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Major  defense  acquisition  programs  historically  have  had  diffi 
culty  controlling  cost,  maintaining  schedule,  and  attaining  performance 
due  to  various  acquisition  strategy  challenges.  Likewise,  with  previous  joint 
aircraft  programs  (F-lll,  V-22,  T-6)  and  now  with  the  F-35  program,  chal¬ 
lenges  associated  with  Balancing  Requirements,  Harnessing  Technology, 
Demanding  Commonality,  Evoking  Concurrency,  and  Encouraging 
Partnering  have  affected  schedule,  cost,  and  performance  outcomes.  ^ 
This  article  summarizes  the  triangulated  research  analysis  on  the 
comparison  of  previous  joint  aircraft  acquisition  programs,  the 
mining  and  coding  of  government  agency/think  tank  reports  and 
scholarly  journals  on  the  F-35  program,  and  the  mining  and  coding 
of  questionnaires  given  to  subject  matter  experts  working 
on  the  F-35  program.  It  argues  that  the  F-35  program  has 
relearned  some  old  lessons  and  learned  some  new  ones, 
and  it  makes  recommendations  on  joint  aircraft 
acquisition  strategies  for  the  future  to  avoid  the 
perception  of  scandal  and  tragedy. 
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The  U.S.  Congress  has  long  been  concerned  about  controlling  schedule 
and  cost  overruns,  and  attaining  expected  performance  in  major  defense 
acquisition  programs  (MDAP;  Blickstein,  Nemfakos,  &  Sollinger,  2013). 
Schedule,  cost,  and  performance  are  the  three  inextricably  linked  pillars 
of  acquisition  (Defense  Acquisition  University  [DAU],  2006). 

The  1986  President's  Blue  Ribbon  Commission  on  Defense  Management, 
referred  to  as  the  Packard  Commission  Report,  determined  that  MDAPs 
take  too  long  to  develop,  cost  too  much,  and  incorporate  obsolete  technol¬ 
ogy  by  the  time  they  are  fielded.  More  recently,  the  2009  Weapon  Systems 
Acquisition  Reform  Act  sought  to  improve  the  likelihood  of  success  of 
MDAPs  by  focusing  on  decisions  and  types  of  strategies  at  their  inception 


(Eide  &  Allen,  2012;  Young  et  al.,  2010).  Out  of  95  known 
MDAPs  from  2006  to  2010,  40%  experienced  delays 


2,457  aircraft  for  the 
United  States,  is  the 
largest  single  global 


defense  program  in  history  at 
$386  billion.  It  accounted  for  $10  billion, 
or  13%,  of  the  cost  growth  in  2011  (GAO,  2011b). 
Recently,  Senator  John  McCain,  chairman  of  the  Senate 
Armed  Services  Committee,  said  the  F-35  program's  record  and 


performance  “has  been  a  scandal  and  a  tragedy''  (Associated  Press,  2016). 

Like  previous  joint  aircraft  programs  (F-lll,  V-22,  and  T-6),  the  F-35  JSF 
program  has  been  subject  to  schedule,  cost,  and  performance  shortcomings 
due  to  acquisition  strategy  challenges  identified  by  U.S.  Government  agen¬ 
cies  and  U.S.  Government  think  tanks  (Blickstein  et  al.,  2013;  GAO,  2012c). 
These  acquisition  strategy  challenges  include  Balancing  Requirements, 
Harnessing  Technology,  Demanding  Commonality,  Evoking  Concurrency, 
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and  Encouraging  Partnering  (Blickstein  et  al.,  2013;  Dunne,  2011;  Ergas, 
2009;  GAO,  2012b;  Wicht  &  Crawley,  2012).  This  article  summarizes  a  2015 
dissertation's  research  methodology  and  lessons  learned  about  strategy 
challenges  from  a  triangulated,  qualitative  case  study  analysis  on  previous 
joint  aircraft  acquisition  programs,  on  governmental  agency  and  think  tank 
reports  and  scholarly  journals  on  the  JSF,  and  on  questionnaire  responses 
by  subject  matter  experts  (SME)  who  were  currently  working  on  the  JSF 
program.  Secretary  of  Defense  (SecDef)  Robert  Gates  (2009)  complained 
that  acquisition  problems  have  been  persistent  and  difficult  despite  congres¬ 
sional  involvement  in  trying  to  resolve  them  (more  than  125  studies  since 
World  War  II  have  produced  no  comprehensive,  effective,  and  permanent 
solutions).  The  goal  of  this  unique  research  is  to  improve  the  cost,  schedule, 
and  performance  of  the  JSF  program  and  other  MDAPs  by  understanding 
better,  and  making  recommendations  on,  the  acquisition  strategy  chal¬ 
lenges  of  Balancing  Requirements,  Harnessing  Technology,  Demanding 
Commonality,  Evoking  Concurrency,  and  Encouraging  Partnering.  The 
research  shows  the  F-35  program  relearned  some  old  lessons  and  learned 
some  new  ones. 

The  F-35  is  a  stealthy,  supersonic,  multirole  fighter  built  by  Lockheed  Martin 
(LM)  in  three  variants  to  penetrate  modern  integrated  air  defenses.  The 
U.S.  Air  Force  (USAF),  several  Partners,  and  Foreign  Military  Sales  (FMS) 
countries  will  fly  the  F-35A  Conventional  Take-Off  and  Landing  (CTOL). 
The  U.S.  Marine  Corps  (USMC),  a  couple  of  Partners,  and  a  possible  FMS 
country  will  fly  the  F-35B  Short  Take-Off/Vertical  Landing  (STOVL).  The 
STOVL  is  the  same  size  as  the  CTOL,  but  the  STOVL  carries  less  fuel  because 
of  the  vertical  lift  fan.  The  U.S.  Navy  (USN)  and  the  USMC  will  fly  the  F-35C 
Carrier  Variant  with  a  much  larger  wing  than  the  other  models  for  carrier 
landing  approach  speeds,  but  it  is  not  being  internationally  marketed. 


Methodology 

A  triangulated,  multilayered,  qualitative  case  study  was  used  to  syn¬ 
thesize  lessons  from  three  lanes  of  analysis:  the  comparison  of  previous 
joint  aircraft  acquisition  programs  (F-lll,  V-22,  and  T-6),  the  mining  and 
coding  of  government  agency  and  think  tank  reports  and  scholarly  journals 
on  the  F-35  program,  and  the  mining  and  coding  of  questionnaire  responses 
from  SMEs  who  were  currently  working  on  the  F-35  program  at  the  time 
of  the  research  in  2015.  The  main  research  question  of  this  study  was  to 
understand  how  acquisition  strategy  challenges  (Harnessing  Technology, 
Demanding  Commonality,  Evoking  Concurrency,  and  Encouraging 
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Partnering)  have  both  helped  and  hindered  joint  aircraft  programs'  sched¬ 
ule,  cost,  and  performance  (the  pillars)  in  terms  of  the  acquisition  strategy's 
original  intent,  its  positive  and  negative  effects  on  the  pillars,  and  what 
improvements  could  be  made. 

Jogulu  and  Pansiri  (2011)  supported  triangulation  over  a  single  approach 
because  it  strengthens  findings  and  inferences  made  for  understanding 
program  management  discipline.  Yin  (1994)  and  Patton  (2002)  believed 
a  deep-rooted  and  multilevel  case  study  analysis  could  help  formulate 
appropriate  relationships  between  phenomena.  As  for  mining  and  coding, 
Patton  (2002)  offered  how  to  use  qualitative  data  analysis  (QDA)  software 
to  categorize  and  to  make  sense  out  of  massive  amounts  of  data.  Finally, 
submitting  a  questionnaire  with  phenomenological  attributes  to  a  heuristic 
group  of  SMEs  produced  high-fidelity  qualitative  analysis  of  the  experi¬ 
ences,  beliefs,  and  perceptions  of  respondents  from  multiple  perspectives 
(Moustakas,  1990;  Shank,  2006).  The  questionnaire  is  phenomenological 
because  the  respondents  answer  the  questions  as  they  perceive  the  situ¬ 
ation  (Moustakas,  1990).  This  approach  is  heuristic  because  the  research 
process  involved  the  experiences  of  the  researcher  in  relation  to  the  ques¬ 
tionnaire  respondents  (Moustakas,  1990).  As  chief  of  the  JSF  Coordination 
and  Training  Office  (JCTO)  at  the  USAF's  Air  Education  and  Training 
Command  (AETC)  for  4  years,  the  researcher  constantly  reflected  on  and 
interpreted  daily  interactions  with  SMEs  from  the  F-35  Joint  Program 
Office  (JPO),  LM,  and  the  USAF's  headquarters,  test  community,  and  train¬ 
ing  operators.  Wacker  (1998)  concluded  that,  when  dealing  with  the  social 
sciences  (including  program  management),  good  recommendations  come 
from  open  questions,  often  applying  the  researcher's  and  SME's  own  expe¬ 
riences,  instead  of  from  scientific,  quantifiable  analysis. 

Each  analysis  lane  is  considered  multilayered  because  three  previous  joint 
aircraft  acquisition  programs  were  reviewed  and  because  three  types  of 
documents  and  three  categories  of  questionnaires  were  mined  and  coded. 
The  F-lll,  V-22,  and  T-6  programs  were  the  most  suitable  comparisons  in 
the  number  of  Services  initially  interested  in  combining  development,  the 
number  of  aircraft  being  procured,  and  the  overall  complexity.  Potential 
sampling  of  previous  joint  aircraft  acquisition  programs  could  have  included 
some  well-known  and  successful  aircraft  acquisition  programs  like  the  F-4, 
F-5,  and  A -7  that  were  used  by  the  Air  Force,  Navy,  and  Marines,  and  several 
other  countries,  but  they  were  originally  developed  by  a  single  Service  first 
(Antill  &  Ito,  2013;  Pike,  2011).  Although  scholarly  journals  and  think  tank 
reports  exist  on  the  F-35  program,  General  Accounting  Office/GAO  and 
Director  of  Operational  Test  and  Evaluation  (DOT&E)  reports  on  the  F-35 
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outnumbered  the  scholarly  journals  and  think  tank  reports  by  44  to  24  in 
this  study.  General  Accounting  Office/GAO  and  DOT&E  reports  on  the  JSF 
go  back  to  2003,  when  some  of  the  acquisition  strategy  challenges  began  to 
emerge  and  provided  vast  amounts  of  numerical  and  statistical  data,  as  well 
as  well-defined  problems  and  recommendations.  The  General  Accounting 
Office/GAO  and  DOT&E  reports  on  the  JSF  were  prepared  by  experts  in  the 
field  of  acquisition;  since  the  reports  were  a  matter  of  public  record,  this 
promoted  validity  and  dependability.  The  original  plan  was  to  e-mail  the 
questionnaire  to  about  50  JSF  SMEs  in  three  categories— 20  out  of 200  from 
the  F-35  JPO  as  the  managers,  10  out  of  100  from  LM  as  the  providers,  and 
20  out  of  200  from  the  U.S.  military  services  as  the  customers.  Leedy  and 
Ormrod  (2009)  recommended  an  unstructured  survey  to  sample  between 
five  and  25  individuals.  The  2015  study  accepted  the  upper  limit  of  25  within 
the  manager  and  customer  categories. 


The  disclosure  that  the  researcher  was  AETC's  chief  of  the  JCTO  during 
the  time  of  the  study  promoted  credibility  and  integrity  with  the  question¬ 
naire  participants,  besides  contributing  to  a  heuristic  research  approach. 
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As  for  construct  validity  on  the  unstructured  questionnaire,  the  2015  study 
followed  the  approach  from  a  questionnaire  in  a  dissertation  by  Uda  (2012). 
The  questionnaire  in  the  2015  study  was  championed  by  the  JPO  deputy 
program  executive  officer  and  vetted  by  the  JPO  security  officer  and  lawyers. 
A  limitation  of  the  2015  study  was  that  only  unclassified  information  was 
used  from  open  source  literature  and  unclassified  answers  from  the  ques¬ 
tionnaire.  One  of  the  toughest  challenges  to  internal  validity  was  the  need 
to  guard  against  the  researcher's  and  respondents'  expectancies  and  biases 
while  being  so  intimately  involved  with  the  F-35  program.  The  participants 
were  not  led  to  foregone  conclusions  through  an  interview;  this  is  why  an 
open  questionnaire  was  used  instead.  The  2015  study  was  delimited  in  scope 
by  using  a  qualitative  methodology  instead  of  a  quantitative  one,  because  the 
large  amount  of  government  data  would  have  been  difficult  to  quantify  any 
actionable  recommendations.  DoD  instructions  and  university  institutional 
review  board  processes  ensured  ethical  standards  were  maintained  with 
respect  to  questionnaire  participants  to  the  point  where  each  participant's 
commander  or  supervisor  approved  contact  by  the  researcher. 


Previous  Joint  Aircraft 
Acquisition  Programs 

Three  previous  joint  aircraft  programs  similar  in  scope  to  the  JSF 
were  compared:  the  1960s  Tactical  Fighter  Experimental  (TFX)  F-lll, 
the  1980s  Joint  Service  Vertical  Takeoff  &  Landing  Experimental  (JVX) 
V-22,  and  the  1990s  Joint  Primary  Aircraft  Training  System  (JPATS)  T-6 
programs.  Examining  the  effect  on  the  pillars  of  acquisition  by  the  previous 
programs'  acquisition  strategy  challenges  acted  as  a  precursor  to  what  the 
JSF  program  has  experienced.  As  previously  discussed,  although  there  were 
successful  aircraft  acquisition  programs  like  the  F-4,  F-5,  and  A- 7,  they 
were  developed  by  a  single  Service  (Antill  &  Ito,  2013;  Pike,  2011).  There 
are  also  some  successful  joint  missile  programs  like  the  Joint  Direct  Attack 
Munition,  but  it  simply  did  not  match  the  scope,  scale,  and  complexities  of 
the  F-lll,  V-22,  and  T-6  programs. 

Tactical  Fighter  Experimental  (TFX)  F-111  Program 

The  TFX  program  introduced  the  multimission  concept  that  would 
affect  the  attack  aircraft  industry  for  the  next  few  decades  (Miller,  1982). 
After  World  War  II,  attack  aircraft  were  developed  for  single  purpose 
missions:  nuclear  strategic  bombing,  tactical  interdiction,  air  superior¬ 
ity,  or  close  air  support  (Miller,  1982).  Furthermore,  SecDef  McNamara 
wanted  to  shift  doctrine  from  massive  nuclear  retaliation  to  a  range  of 
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conventional  options  (Coulam,  1977). 

Still,  the  USAF  wanted  a  follow-on 
F-105  fighter-bomber  for  the  deliv¬ 
ery  of  internally  carried  tactical 
nuclear  missiles  ( TFX Contract 
Investigation,  1963).  The  USN 
wanted  an  air-to-air  mis¬ 
sile  carrier  to  identify  and 
shoot  down  enemy  planes  at 
extended  ranges  from  their 
carriers  (Coulam,  1977).  The 
USAF  and  the  USN  could 
agree  only  on  a  swing-wing, 
two-seat,  and  twin-engine 
design  (Art,  1969).  The  USAF 
wanted  a  tandem-seat  aircraft 
(pilot  in  front  and  weapon  system 
operator  behind)  for  low-level  pene¬ 
tration  ground-attack,  while  the  USN 
wanted  a  shorter,  high-altitude  interceptor 
with  side-by-side  seating  to  allow  the  pilot  and 
radar  intercept  officer  to  share  the  radar  display  (Miller,  1982).  Coulam 
(1977)  concluded  that  directly  competing  requirements  were  inevitably 
traded  off,  never  fully  meeting  either  Service's  requirements. 


The  TFX  program  resulted  in  the  F-lll,  produced  by  General  Dynamics  (GD), 
serving  primarily  as  a  supersonic,  medium-range  interdictor  and  tactical 
attack  aircraft  that  later  filled  the  roles  of  strategic  bomber  and  electron¬ 
ic-warfare  aircraft  (Logan,  1998).  It  first  entered  service  with  the  USAF  in 
1967  and  then  with  the  Royal  Australian  Air  Force  in  1973  (Logan,  1998). 
The  F-lll  featured  new  variable-geometry  wings  for  high-  and  low-speed 
flight  with  leading-edge  slats  and  double-slotted  flaps  over  its  full  length  to 
create  more  lift  for  relatively  short  runway  use.  It  also  had  afterburning 
turbofan  engines  and  automated  terrain-following  radar  for  low-level,  high¬ 
speed  flight  (Logan,  1998).  A  major  failing  of  the  TFX  program  was  that  it 
asked  too  much  of  technology  too  soon  (TFX  Contract  Investigation,  1970). 
SecDef  McNamara  and  the  Services  looked  to  GD  to  solve  all  of  the  issues 
with  new  innovations  ( TFX  Contract  Investigation,  1963).  Although  vari¬ 
able-geometry  wings  worked  as  advertised,  the  poor  performance  of  the 
Mark  II  low-level  avionics,  T-30  turbofan  engines,  and  variable  inlets  would 
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plague  the  military  services  ( TFX  Contract  Investigation,  1970).  The  program 
also  experienced  a  25%  concurrency  rate— 141  out  of 547  total  USAF  F-llls 
needed  retrofits  (Coulam,  1977;  Richey,  2005). 


Most  USAF  programs  in  the  1950s  exceeded  their 
costs  by  100  to  200%  and  their  schedules  by  36  to  50% 
(Summers,  1965). 


Cost  estimates  also  concerned  SecDef  McNamara  ( TFX  Contract 
Investigation,  1970).  He  recognized  that  the  Services  had  limited  resources 
and  funding  from  Congress,  so  in  order  to  get  more  from  their  budgets, 
they  would  encourage  bids  that  were  unrealistically  low  ( TFX  Contract 
Investigation,  1963).  Once  the  Services  had  congressional  support  and 
dedication  to  continue  the  program,  costs  predictably  rose,  but  the  Services 
were  likely  to  get  additional  funds  to  finish  the  program  (TFX  Contract 
Investigation,  1963).  Most  USAF  programs  in  the  1950s  exceeded  their 
costs  by  100  to  200%  and  their  schedules  by  36  to  50%  (Summers,  1965). 
McNamara  passionately  drove  for  a  single  aircraft  to  meet  the  needs  of  both 
the  USAF  and  USN,  expecting  a  high  degree  of  commonality  between  the 
two  versions  (TFX  Contract  Investigation,  1963).  GD  planned  to  reduce  cost 
and  risk  by  adding  Grumman  and  P&W  as  aircraft  development  partners 
(TFX  Contract  Investigation,  1963).  GD  was  also  able  to  reduce  predicted 
unit  price  by  courting  the  U.K.  early  as  a  partner  to  buy  several  of  a  special¬ 
ized  variant  of  the  F-lll,  expecting  commonality  to  save  money  (Hunter, 
1998;  Logan,  1998).  SecDef  McNamara  chose  GD  over  Boeing  for  more 
realistic  cost  estimates,  but  the  Services  were  guilty  of  assuming  high 
expectations  on  technical  performance  (TFX  Contract  Investigation,  1970). 
In  the  end,  the  TFX  program  had  a  100%  cost  overrun  and  a  30  to  40%  sched¬ 
ule  overrun  (TFX  Contract  Investigation,  1970),  and  only  seven  F-lllBs  were 
built  for  the  USN  for  test  purposes  before  they  cancelled  out  of  the  program 
(General  Accounting  Office,  1973). 

Joint  Service  Vertical  Takeoff  and  Landing  Experimental 
(JVX)  V- 22  Program 

The  V-22  is  a  multimission,  tilt-rotor  aircraft  with  both  a  vertical  take¬ 
off  and  landing  (VTOL)  capability  like  a  helicopter  and  a  fixed-wing  aircraft 
capability,  achieved  by  tilting  its  wing-mounted  rotors  to  act  as  propellers 
(General  Accounting  Office,  1990, 1994;  Whittle,  2010).  The  JVX  program 
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started  in  1981  to  meet  joint  Service  requirements  that  would  satisfy  USMC 
medium-lift  assault,  USN  search  and  rescue,  and  USAF  long-range  spe¬ 
cial  operations  (General  Accounting  Office,  1990,  1994,  1997).  The  DoD 
awarded  Bell  Helicopter  and  Boeing  Helicopters  a  development  contract 
in  1983  (Whittle,  2010).  The  U.S.  Army  (USA)  planned  to  use  the  USMC's 
assault  requirements  for  its  medium  cargo  lift  and  medical  evacuation  needs 
(General  Accounting  Office,  1986).  When  the  first  V-22  rolled  out  in  1988, 
the  USA  had  already  left  the  program  for  good  to  focus  its  budget  on  more 
immediate  aviation  programs  (Whittle,  2010). 

The  JVX's  original  program  cost  estimates  changed  significantly,  and  its 
development  process  was  long  and  controversial  (Whittle,  2010).  One  of 
the  USN's  cost-saving  strategies  for  the  USMC's  MV-22s  included  a  high 
level  of  concurrent  development  (General  Accounting  Office,  1994).  The 
General  Accounting  Office  (1994)  warned  that  such  concurrency  involved 
high  risk  that  eventually  required  rescheduling  and  spending  on  increased 
overtime.  The  V-22  began  flight  testing  in  1989  and  started  design  alter¬ 
ations  immediately  (Whittle,  2010).  The  complexity  and  difficulties  of  being 
the  first  tilt-rotor  intended  for  military  service  in  the  world  led  to  many 
years  of  development  (Whittle,  2010).  The  JVX  program  faced  opposition 
in  the  Senate  in  1989,  surviving  two  separate  motions  that  both  could  have 
resulted  in  program  cancellation  (Whittle,  2010).  The  full-scale  devel¬ 
opment  contract  was  even  terminated  once  in  October  1992  because  Bell 
and  Boeing  failed  to  assemble  all  six  flight-test  aircraft,  failed  to  perform 
all  planned  drop  and  fatigue  tests,  and  did  not  complete  all  flight  testing 
(General  Accounting  Office,  1994). 

The  V-22  program  entered  full-rate  production  without  mature  manu¬ 
facturing  processes  that  required  a  redesign  and  retrofit  of  the  hydraulic 
and  electric  system  and  led  to  a  Nunn-McCurdy  breach  in  2001  (General 
Accounting  Office,  2003).  Although  the  USMC  began  crew  training  for 
the  MV-22  Osprey  in  2000,  it  did  not  declare  initial  operational  capability 
(IOC)  until  2007  (Whittle,  2010).  The  Osprey's  other  current  operator, 
the  USAF,  declared  IOC  in  2009  with  their  CV-22  version  of  the  tilt-rotor 
(Whittle,  2010).  Although  12  MV-22s  deployed  to  Iraq  in  January  2009, 
and  confirmed  there  that  the  MV-22's  enhanced  speed  and  range  enabled 
personnel  and  internally  carried  cargo  to  be  transported  faster  and  farther 
than  by  legacy  helicopters  (GAO,  2009),  almost  30  years  had  passed  from 
program  inception  to  real-world  execution.  During  that  time,  V-22  costs 
have  risen  sharply  above  initial  projections— 1986  estimates  (stated  in  fiscal 
year  2009  dollars)  that  the  program  would  build  nearly  1,000  aircraft  in  10 
years  at  $37.7  million  each  have  shifted  to  fewer  than  500  aircraft  at  $93.4 
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million  each— a  procurement  unit  cost  increase  of  148%,  while  research, 
development,  testing,  and  evaluation  costs  increased  over  200%  (Gertler, 
2011).  Even  after  the  Department  of  State  approved  Japan  in  2015  for 
acquisition  of  up  to  17  V-22B  Block-C  Ospreys  and  all  the  logistical  support, 
Japan  deferred  their  purchase  indefinitely  due  to  their  budget  restraints  and 
the  predicted  increased  costs  to  maintain  the  complicated  weapon  system 
(McCullough,  2015). 


Joint  Primary  Aircraft  Training  System  (JPATS)  T-6  Program 

In  1988,  the  USAF  and  the  USN  worked  together  on  the  DoD  Trainer 
Aircraft  Masterplan  and  formed  the  JPATS  program  to  modernize  their 
training  aircraft  fleets  and  methods  of  primary  flight  training  (AETC,  2010). 
Once  the  USAF  and  USN  finally  agreed  on  tandem  cockpits  (the  instructor 
behind  the  student)  and  the  anthropometries  of  the  ejection  seat  to  allow 
more  women  into  flight  training,  the  USAF  and  USN  settled  on  the  commer¬ 
cial-off-the-shelf  (COTS)  Hawker  Beechcraft  (formerly  Raytheon  Aircraft 
Company)  Pilatus  PC-9  aircraft  (AETC,  2010). 

Military-unique  design  requirements  off  the  COTS  baseline  grew  from 
about  5%  when  the  program  entered  limited  production  in  1995  to  almost 
70%  by  the  early  2000s  (General  Accounting  Office,  2003).  Furthermore, 
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balancing  requirements  between  the  USAF  and  USN  led  to  a  22%  heavier 
aircraft  than  its  original  COTS  version  (Gantt,  2002).  It  took  7  years  from 
the  establishment  of  the  DoD  Trainer  Aircraft  Masterplan  and  JPATS  pro¬ 
gram  in  1988  to  aircraft  coming  off  the  assembly  line  in  1995  for  a  relatively 
simple  mission  (AETC,  2010).  Twelve  years  into  production  in  2007,  JPATS 
experienced  a  “critical”  Nunn-McCurdy  breach,  exceeding  50%  cost  growth 
from  its  baseline  (GAO,  2007).  A  DoD  review  concluded  that  the  cost  growth 
was  attributed  to  changes  in  government  requirements,  and  the  remainder 
was  due  to  immature  and  unchecked  manufacturing  processes  (GAO,  2007). 
Once  JPATS  was  rebaselined  in  2008  for  cost  and  schedule  as  required  by 
Nunn-McCurdy,  several  foreign  countries  signed  contracts  directly  with 
Hawker  Beechcraft  via  direct  commercial  sales  vice  FMS  (DoD,  2012). 

Assessment  of  Previous  Joint  Aircraft  Programs 

The  TFX  F-lll,  JVX  V-22,  and  JPATS  T-6  programs  experienced  sim¬ 
ilar  instances  of  acquisition  strategy  challenges  in  relation  to  the  pillars  of 
acquisition.  Several  specific  instances  were  tabularized  in  the  dissertation 
study  this  article  is  based  upon.  Each  program  had  at  least  one  acquisition 
strategy  challenge  affecting  more  than  one  acquisition  pillar  at  the  same 
time  or  one  of  the  challenges  worked  in  tandem  with  another  to  affect  either 
one  or  more  of  the  pillars.  Figure  1  begins  to  answer  the  research  subques¬ 
tions  concerning  acquisition  strategy  challenges'  positive  and  negative 
effects  on  the  pillars  of  acquisition  just  by  looking  at  the  up  or  down  arrows 
beside  each  factor  below  each  pillar  column  in  relation  to  the  acquisition 
strategy  challenge  on  the  left.  For  example,  at  the  top  of  Figure  1  at  the  inter¬ 
section  of  “Cost”  and  “Balancing  Requirements,”  a  down  arrow  representing 
reduced  cost  stands  next  to  the  comment  “Combine  programs  to  get  one  air¬ 
craft,”  while  an  up  arrow  stands  next  to  the  comment  “Meet  multi-Services' 
missions  and  operating  environments.”  The  important  message  that  Figure 
1  is  trying  to  convey  is  that  acquisition  strategy  challenges  are  not  mutually 
exclusive,  exemplified  by  having  the  color  of  one  strategy  affecting  one  or 
more  pillars  of  another  strategy.  For  example,  Balancing  Requirements 
(red)  and  Demanding  Commonality  (yellow)  significantly  overlap  and  affect 
each  other.  Cost  and  performance  are  highlighted  by  the  opposite  color  of 
the  strategy  challenge  because  it  is  difficult  to  meet  the  Services'  individual 
mission  needs  and  preferences.  Likewise,  Harnessing  Technology  (blue) 
and  Evoking  Concurrency  (green)  overlap,  affecting  mostly  performance 
and  schedule  because  there  can  never  be  enough  engineering  and  modeling 
of  new  technology  when  that  new  technology  is  already  in  the  production 
line  with  few  actual  test  flights. 
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FIGURE  1.  PREVIOUS  JOINT  AIRCRAFT  PROGRAMS’ 
INTERRELATIONSHIPS  BETWEEN  PILLARS  OF  ACQUISITION  AND 
ACQUISITION  STRATEGY  CHALLENGES 


Pillars  of  Acquisition 

Technical 

Schedule  Cost  Performance 

▲  =  Lengthens  ▲  =  Increases  ▲  =  Keeps  or 

T  =  Shortens  T  =  Decreases  Improves 

T  =  Reduces 

Balancing 

Requirements 

▲  Drive  consensus 

▲  Rushing  to  meet 
milestones 

▼  Combine  programs  to 

get  one  aircraft 

A  Meet  multi-Services’  ▼  Contradictory  aerody- 

missions  and  operat-  namic  requirements 

ing  environments  ▼  Concessions  even- 

A  Mission/requirements  tually  made  by  one 

additions  or  changes  Service 

A  Contradictory  aerody¬ 

namic  requirements 

Harnessing 

Technology 

▲  Immature  manufac¬ 
turing  processes 

▲  Not  previously  at¬ 
tempted 

▲  Immature  technology 

▲  Mature  technology  or 
commercial-off-the- 
shelf 

A  Not  previously 
attempted 

A  Immature  technology 

A  Need  for  asymmetric 
advantage 

▼  Not  previously  at¬ 
tempted 

▼  Immature  technology 

A  Mature  technology  or 

commercial-off-the- 

shelf 

Demanding 

Commonality 

A  More  stakeholders 
require  more  time  to 
decide 

▼  One  aircraft  to  meet 
needs  of  two  or  more 
Services 

A  %  of  common  parts 
decreases  as  time 
passes 

▼  As  Service  autonomy 
is  threatened 

Evoking 

Concurrency 

T  Development,  produc¬ 
tion,  testing  at  nearly 
the  same  time 

A  Immature  manufactur¬ 
ing  processes 

▼  Development,  produc¬ 
tion,  testing  at  nearly 
the  same  time 

A  Retrofits  if  problems 
arise  during  testing 

A  Immature  technology 

A  Mature  technology  or 
commercial-off-the- 
shelf 

▼  If  not  enough  testing 
or  the  right  type  of 
testing— simulated  or 
live  flight 

▼  Retrofits  if  problems 
arise  during  testing 

Encouraging  Inter¬ 
national  Partnering 

A  More  countries  sooner 
have  more  say 

T  More  countries  buying 
▼  More  countries  sooner 

T  Desire  for  coalition 
interoperability 

Note.  Specific  instances  of  strategy  effect  on  pillars  from  TFX  F-111,  JVX  V- 22,  and  JPATS 
T-6  programs  were  generalized  from  the  dissertation.  Color  scheme  demonstrates  that 
acquisition  strategy  challenges  are  not  mutually  exclusive. 
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Government  Agency  and  Think  Tank 
Reports  and  Scholarly  Journals  on  the  JSF 

Government  agency  and  think  tank  reports  and  scholarly  journals  on 
the  JSF  program  were  mined  and  coded  for  acquisition  strategy  challenges 
and  pillars  of  acquisition,  and  to  see  whether  and  how  recommendations 
were  executed.  Although  ATLAS.ti  QDA  software  assisted  in  mining  and 
coding,  it  was  mostly  used  for  cataloging  and  for  determining  strong  asso¬ 
ciations  between  acquisition  strategy  challenges  and  pillars  of  acquisition, 
and  within  one  another  by  counting  co-occurrences  and  determining  c-co- 
efficients  (Friese,  2013).  Friese  (2013)  recommended  that  strong  associations 
should  be  for  c-coefficients  >  0.08.  The  2015  study  addressed  each  strong 
association  as  long  as  there  were  two  co-occurrences. 


Government  agency  and  think  tank  reports  and 
scholarly  journals  on  the  JSF  program  were  mined 
and  coded  for  acquisition  strategy  challenges  and 
pillars  of  acquisition,  and  to  see  whether  and  how 
recommendations  were  executed . 


Within  the  “super-family”  code  of  acquisition  strategy  challenges,  the 
following  coding  families  were  established:  Balancing  Requirements, 
Harnessing  Technology,  Demanding  Commonality,  Evoking  Concurrency, 
and  Encouraging  Partnering.  Although  the  research  questionnaire  asked 
three  questions  on  each  strategy,  there  were  really  four  coding  components 
for  each  family  code:  “original  intent,  negative  effects,  positive  effects,  and 
improvement  recommendations.”  Within  ATLAS.ti,  they  were  coded  as 
Balancing  Requirements  Intent  (BRI),  Balancing  Requirements  Negative 
(BRN),  Balancing  Requirements  Positive  (BRP),  Balancing  Requirements 
Recommendation  (BRR),  Technology  Intent  (TI),  Technology  Negative  (TN), 
Technology  Positive  (TP),  Technology  Recommendation  (TR),  Commonality 
Intent  (CMI),  Commonality  Negative  (CMN),  Commonality  Positive 
(CMP),  Commonality  Recommendation  (CMR),  Concurrency  Intent  (CCI), 
Concurrency  Negative  (CCN),  Concurrency  Positive  (CCP),  Concurrency 
Recommendation  (CCR),  Partnering  Intent  (PTI),  Partnering  Negative 
(PTN),  Partnering  Positive  (PTP),  and  Partnering  Recommendation  (PTR). 
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Within  the  “super-family”  pillars  of  acquisition  code ,  the  coding  families  of 
“cost,  schedule,  and  performance”  were  created.  After  the  literature  review 
and  pre-reading  the  respondents'  questionnaires,  it  was  appropriate  to  have 
five  coding  components  for  each  pillar  family  code  to  separate  the  reasons 
from  other  statements.  For  cost,  the  codes  were  Cost  Increased  (Cl)— 
bad,  Cost  Increase  Reason  (CIR)— why,  Cost  Decreased  (CD)— good,  Cost 
Decrease  Reason  (CDR)— why,  and  Cost  Recommendation  (CR).  For  sched¬ 
ule,  the  codes  were  Schedule  Lengthened  (SL)— bad,  Schedule  Lengthened 
Reason  (SLR)— why,  Schedule  Shortened  (SS)— good,  Schedule  Shortened 
Reason  (SSR)— why,  and  Schedule  Recommendation  (SR).  For  performance, 
the  codes  were  Performance  Reduced  (PRD)— bad,  Performance  Reduced 
Reason  (PRR)— why,  Performance  Improved  (PI)— good,  Performance 
Improved  Reason  (PIR)— why,  and  Performance  Recommendation  (PR). 

Table  1  shows  the  breakdown  of  family  codes  by  the  type  of  govern¬ 
ment  report  or  scholarly  journal.  As  previously  mentioned,  Government 
Accountability  Office  (GAO)  and  DOT&E  reports  on  the  F-35  outnumbered 
the  scholarly  journals  and  think  tank  reports  by  44  to  24  in  this  study.  Out 
of  68  total  documents,  the  GAO  was  the  largest  represented  group,  with  33 
total  documents  between  Selected  Acquisition  Reports  and  specific  reports 
on  the  F-35.  Although  the  GAO  had  more  documents  than  the  scholarly  jour¬ 
nals,  the  GAO  usually  had  the  same  researchers  investigating  and  writing 
the  reports  for  several  years,  so  there  is  a  reputation  of  expertise  that  could 
not  be  ignored.  With  only  11  documents,  DOT&E  was  coded  the  most,  and 
its  highest  percentage  of  codes  went  to  the  Harnessing  Technology  fam¬ 
ily  of  codes.  DOT&E  had  numerous  Technology  Recommendation  (TR) 
codes  to  adjudicate  the  large  number  of  Technology  Negative  (TN)  codes. 
The  GAO  also  coded  the  Harnessing  Technology  family  of  codes  pretty 
often,  but  it  highlighted  mostly  TN  aspects,  some  Technology  Positive  (TP) 
aspects,  and  very  few  Technology  Recommendations  (TR)  as  compared  to 
DOT&E.  Think  tank  coding  and  scholarly  journal  coding  were  evenly  dis¬ 
tributed  among  the  strategies  and  pillars,  but  scholarly  journals  coded  the 
Encouraging  Partnering  family  of  codes  the  most.  It  did  not  go  unnoticed 
that  the  government  agency/think  tank  reports  coded  more  negatively 
across  most  strategies;  whereas  scholarly  journals  only  slightly  favored 
negative  codes,  but  in  terms  of  broad  program  management.  There  were  234 
passages  coded  as  recommendations  from  the  governmental  agency/think 
tank  reports  and  scholarly  journals  out  of  798  total  codes. 
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TABLE  1.  REPORTSVJOURNALS’  FAMILY  CODE  BREAKDOWN  1 

DOT&E  (11) 

GAO  SAR  (13) 

GAO  F-35  (20) 

Think  Tanks  (8) 

Journals  (16) 

Totals  (68) 

Balancing  Requirements  Intent  (BRI) 

5 

0 

1 

4 

0 

10 

Balancing  Requirements  Negative  (BRN)  12 

4 

0 

7 

2 

25 

Balancing  Requirements  Positive  (BRP) 

8 

1 

0 

0 

1 

10 

Balancing  Requirements 

R 

o 

o 

A 

1 

Recommendation  (BRR) 

u 

z 

Hr 

1 

1  o 

Balancing  Requirements  Sub-total 

30 

5 

3 

14 

4 

56 

Technology  Intent  (Tl) 

8 

2 

0 

3 

1 

14 

Technology  Negative  (TN) 

39 

38 

19 

10 

8 

114 

Technology  Positive  (TP) 

16 

15 

3 

0 

2 

36 

Technology  Recommendation  (TR) 

61 

1 

8 

7 

5 

82 

Harnessing  Technology  Sub-total 

112 

55 

30 

20 

15 

232 

Commonality  Intent  (CMI) 

0 

2 

0 

5 

4 

11 

Commonality  Negative  (CMN) 

3 

0 

1 

11 

7 

22 

Commonality  Positive  (CMP) 

1 

1 

0 

0 

7 

9 

Commonality  Recommendation  (CMR) 

1 

0 

1 

3 

5 

10 

Demanding  Commonality  Sub-total 

5 

3 

2 

17 

21 

48 

Concurrency  Intent  (CCI) 

0 

3 

0 

2 

1 

6 

Concurrency  Negative  (CCN) 

13 

17 

15 

2 

3 

50 

Concurrency  Positive  (CCP) 

0 

3 

1 

0 

0 

4 

Concurrency  Recommendation  (CCR) 

11 

2 

6 

0 

4 

23 

Evoking  Concurrency  Sub-total 

24 

23 

22 

4 

8 

81 

Partnering  Intent  (PTI) 

0 

0 

0 

3 

6 

9 

Partnering  Negative  (PTN) 

0 

1 

0 

4 

8 

13 

Partnering  Positive  (PTP) 

0 

0 

0 

1 

9 

10 

Partnering  Recommendation  (PTR) 

2 

1 

0 

2 

0 

5 

Encouraging  Partnering  Sub-total 

2 

1 

0 

8 

19 

30 
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TABLE  1.  REPORTS’/JOURNALS’  FAMILY  CODE  BREAKDOWN, 

CONTINUED 

DOT&E  (11) 

GAO  SAR  (13) 

GAO  F-35  (20) 

Think  Tanks  (8) 

Journals  (16) 

Totals  (68) 

Cost  Increase  (Cl)— bad 

0 

8 

10 

6 

5 

29 

Cost  Increase  Reason  (CIR)— why 

4 

16 

8 

8 

10 

46 

Cost  Decrease  (CD)— good 

0 

2 

1 

0 

1 

4 

Cost  Decrease  Reason  (CDR)— why 

0 

3 

3 

2 

5 

13 

Cost  Recommendation  (CR) 

9 

3 

15 

13 

7 

37 

Cost  Sub-total 

12 

30 

36 

19 

28 

126 

Schedule  Lengthened  (SL)— bad 

7 

12 

9 

2 

2 

32 

Schedule  Lengthened  Reason  (SLR)— 
why 

13 

20 

8 

2 

2 

45 

Schedule  Shortened  (SS)— good 

0 

1 

1 

0 

0 

2 

Schedule  Shortened  Reason  (SSR)— why 

3 

3 

2 

0 

0 

8 

Schedule  Recommendation  (SR) 

21 

3 

5 

1 

2 

32 

Schedule  Sub-total 

42 

36 

25 

5 

6 

115 

Performance  Reduced  (PRD)— bad 

16 

4 

3 

4 

2 

29 

Performance  Reduced  Reason  (PRR)— why 

10 

7 

0 

1 

4 

22 

Performance  Improved  (PI)— good 

8 

8 

8 

0 

1 

26 

Performance  Increased  Reason  (PIR)— why 

2 

3 

2 

0 

2 

9 

Performance  Recommendation  (PR) 

20 

1 

3 

5 

3 

32 

Performance  Sub-total 

52 

21 

17 

10 

12 

112 

Grand  Totals 

279 

174 

135 

97 

113 

798 

Note.  (X)  =  number  of  documents,  DOT&E  =  Director  of  Operational  Test  and  Evaluation, 
GAO  =  Government  Accountability  Office,  SAR  =  Selected  Acquisition  Reports 
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Table  2  provides  insight  into  the  opening  (O),  working  (W),  and  closing 
(C)  timing  of  80  official  recommendations  made  by  the  DOT&E,  the  GAO, 
and  the  DoD  Inspector  General  directly  to  the  JSF  JPO.  Most  of  DOT&E's 
closed  recommendations  to  the  JPO  were  divided  between  improvements 
for  flight  test  strategy,  planning  and  realism,  and  with  Autonomic  Logistics 
Information  System  (ALIS)  testing.  DOT&E's  open  recommendations  deal 
with  mission  data  load  test  integration,  the  helmet  mounted  display  sys¬ 
tem,  fueldraulics*  survivability,  VSim  verification,  aircraft  repair  times, 
Block  2B  weapons  delivery  accuracy,  and  F-35B  STOVL  fielding  concerns. 
Most  of  GAO's  closed  recommendations  to  the  JPO  dealt  with  cost  and 
reschedule  baselining,  maintaining  expected  funding,  monitoring  software, 
and  F-35B  progress.  GAO's  open  recommendations  deal  with  executing 
knowledge-based,  evolutionary  acquisitions  and  limiting  production  strat¬ 
egies.  The  open  recommendations  by  the  Department  of  Defense  Inspector 
General  (DoD  IG)  to  the  JPO  deal  with  establishing  quality  assurance  pro¬ 
grams  over  the  contractor,  which  the  JPO  disagrees  with  because  it  does  not 
have  the  resources  or  the  responsibility  to  perform  this  through  the  supply 
chain  (DoD  IG,  2013). 
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TABLE  2.  TRACKING  RECOMMENDATIONS  TO  THE  JPO 


Recommendations 

m 

o 

o 

CM 

2006 

N 

o 

o 

CM 

2008 

0> 

o 

o 

CM 

2010 

2011 

2012 

2013 

2014 

2015 

DOT&E  05— Have  data  collection  and  mission 
replay  to  evaluate  mission  effectiveness 

O 

c 

DOT&E  05— Test  and  Evaluation  Master  Plan 
(TEMP)  should  identify  shortfalls  in  opposing 
force/threats 

o 

w 

w 

w 

w 

c 

DOT&E  05— Align  requirements  for  each  block 
aircraft 

o 

c 

DOT&E  05— Develop  predictive  model  on 
engine  performance  after  “quick  dump”  fuel 
ingestion 

o 

w 

w 

c 

DOT&E  05— Reduce  fuel  ingestion  vulnera¬ 
bility  by  improving  fuel  bladders  around  inlet 
ducts 

o 

c 

DOT&E  06— Update  issues  from  recent  opera¬ 
tional  assessment 

o 

c 

DOT&E  06— Consider  doing  Initial  Operational 
Test  and  Evaluation  (IOT&E)  earlier  with  oper¬ 
ationally  representative  aircraft 

o 

c 

DOT&E  06— Follow  May  06  Defense  Acquisi¬ 
tion  Board  ideas  for  Partner  testing 

o 

c 

DOT&E  06— Fund  adequate  full-scale  aerial 
target  to  confirm  operational  effectiveness 

o 

c 

DOT&E  06— Conduct  full-up,  live  system-level 
live-fire  ballistic  tests  on  F-35  to  determine 
vulnerability 

o 

w 

c 

DOT&E  07— Retain  last  two  system  develop¬ 
ment  and  demonstration  (SDD)  aircraft 

o 

c 

DOT&E  07— Ensure  labs  are  resourced  to  exe¬ 
cute  verification  strategy  and  to  surge 

o 

w 

w 

w 

c 

DOT&E  07— Develop  metrics  for  verification 
strategy 

o 

w 

w 

c 

DOT&E  07— Develop  entrance  criteria  for 

IOT&E 

o 

w 

w 

c 

DOT&E  07— Reinstate  dry  bay  engine  fire 
suppression 

o 

c 

DOT&E  07— Reinstate  engine  fuel  ingestion 
suppression  liner 

o 

c 

DOT&E  07— Add/Improve  cockpit  warning 
lights  to  F-35B  for  ballistic  damage  before 
vertical  landing 

o 

w 

w 

w 

w 

w 

w 

c 

DOT&E  07— Retain  engine  fueldraulics  and 
liquid  cooling  shutoff  valves 

o 

c 

DOT&E  08— Add  resources  to  flight  testing  in 
FY09-11 

o 

w 

w 

c 

DOT&E  08— Explain  all  test  changes  to  DOT&E 

o 

w 

c 
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TABLE  2.  TRACKING  RECOMMENDATIONS  TO  THE  JPO,  CONTINUED 


Recommendations 

m 

o 

o 

CM 

2006 

N 

O 

O 

CM 

CO 

o 

o 

CM 

0> 

o 

o 

CM 

2010 

2011 

CM 

o 

CM 

2013 

2014 

2015 

DOT&E  08— Initiate  a  Test  Plan  Working  Group 

O 

w 

c 

DOT&E  08— Stabilize  production  and  deliver¬ 
ies  of  systems  needed  for  OT&E  and  training 

o 

w 

w 

c 

DOT&E  08— Complete  third  iteration  of  the 
TEMP 

o 

w 

c 

DOT&E  08— Improve  verifications  simulator 
(VSim)  to  meet  adequate  verification  testing 

o 

w 

w 

c 

DOT&E  08— Restore  capability  to  minimize 
fueldraulics  spillage  from  threat-induced 
damage 

o 

w 

w 

w 

w 

w 

? 

DOT&E  09— Focus  delivery  efforts  on  SDD 
aircraft 

o 

w 

c 

DOT&E  09— By  an  Operational  Test  Review 
Team,  review  IOT&E  test  plan  for  Block-3 
aircraft  systems 

o 

c 

DOT&E  09— Have  more  transparent  contract 
negotiations  for  Block-3  test  aircraft 

o 

c 

DOT&E  09— Verify,  validate,  and  accredit  test 
labs 

o 

w 

c 

DOT&  E  10— Assure  new  flight  test  schedule 
is  realistic 

o 

w 

w 

w 

? 

DOT&E  10— Evaluate  flight  test  schedule, 
executed  versus  planned 

o 

w 

? 

DOT&E  10— Determine  impact  of  technical 
issues  of  helmet-mounted  display 

o 

w 

? 

DOT&E  10— Assure  software  integration  is  in 
flight  test 

o 

c 

DOT&E  10— Verify/validate  msn  data  loads 
(MDLs) 

o 

w 

? 

DOT&E  10— Redesign  On-board  Inert  Gas 
Generating  System  (OBIGGS)  to  maintain 
oxygen  levels  below  where  fire  can  be 
sustained 

o 

w 

w 

w 

w 

- 

DOT&E  11— Use  event-driven  criteria  to  begin 
fit  ops 

o 

? 

DOT&E  11— Test  transonic  buffeting 

o 

? 

DOT&E  11— Determine  impacts  of  late 
structural  durability  testing 

o 

? 

DOT&E  11— Improve  spare/resupply  for  flight 
test 

o 

? 

DOT&E  11— Survey  test  plans  for  certifications 
by  outside  government  agencies 

o 

? 

DOT&E  12— Make  corrections  to  Version  4  of 
TEMP 

o 

? 
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TABLE  2.  TRACKING  RECOMMENDATIONS  TO  THE  JPO,  CONTINUED 


Recommendations 

2005 

2006 

2007 

00 

o 

o 

Ol 

0> 

o 

o 

w 

2010 

2011 

N 

O 

Ol 

2013 

2014 

2015 

DOT&E  12— Conduct  testing  on  Autonomic 
Logistics  Information  System  (ALIS) 

o 

w 

? 

DOT&E  12— Make  operational  test  (OT)  air¬ 
craft  fully  production-representative 

o 

c 

DOT&E  12— Ensure  contractor  meeting  VSim 
requirements 

o 

w 

? 

DOT&E  12— Assure  integrated  VSim  and  MDL 
testing 

o 

c 

DOT&E  12— Continue  PAO  shutoff  valve 
redesign 

o 

w 

? 

DOT&E  12— Consider  removing  fueldraulic 
fuses 

o 

c 

DOT&E  12— Consider  keeping  dry  bay  fire  ex¬ 
tinguisher  for  the  Integrated  Power  Pack  only 

o 

w 

? 

DOT&E  12— Determine  ballistic  event  survival 
time 

o 

w 

? 

DOT&E  13— Account  for  historical  growth  of 
flight  test 

o 

? 

DOT&E  13— Get  VSim  data  for  SDD  flight  test 

o 

? 

DOT&E  13— Track  metrics  for  software  sta¬ 
bility 

o 

? 

DOT&E  13— Determine  viability  of  putting 
270-volt  power  on  28-volt  signal  bus 

o 

c 

DOT&E  13— Track  low  observable  (LO)  and 
non-LO  repair  times 

o 

c 

DOT&E  14-Update  IOT&E  Schedules 

o 

- 

DOT&E  14— Complete  MDL  testing  before 
flight  test 

o 

- 

DOT&E  14— Complete  Block  2B  weapon  deliv¬ 
ery  accuracy 

o 

- 

DOT&E  14— Require  contractor  to  do  finite 
element  analysis  on  F-35B  bulkhead 

o 

- 

DOT&E  14— Resource  Block  3  VSim  ade¬ 
quately 

o 

- 

DOT&E  14— Accelerate  joint  technical  data 
(JTD)  verification  for  fielded  F-35Bs 

o 

- 

DOT&E  14— Extend  decontamination  tests 

o 

- 

GAO  05— Establish  an  executable  program 
consistent  with  best  practices  and  DoD  poli¬ 
cy  regarding  knowledge-based,  evolutionary 
acquisition 

o 

w 

w 

w 

w 

w 

w 

w 

w 

w 

- 

GAO  07— Limit  annual  production  quantities 
to  no  more  than  24  aircraft  per  year  until 
each  variant’s  basic  flying  qualities  have  been 
demonstrated 

o 

w 

w 

w 

w 

w 

w 

w 

- 

122 


Defense ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


January  2017 


TABLE  2.  TRACKING  RECOMMENDATIONS  TO  THE  JPO,  CONTINUED 


Recommendations 

m 

o 

o 

M 

2006 

2007 

2008 

0> 

o 

o 

Ol 

2010 

2011 

2012 

2013 

2014 

o 

M 

GAO  08— Revisit  and  revise  the  Mid-Course 
Risk  Reduction  Plan  to  address  concerns 
about  testing,  use  of  management  reserves, 
and  manufacturing 

o 

w 

w 

w 

w 

c 

GAO  08— Improve  JSF  cost  estimate  reliability 

o 

w 

w 

w 

w 

c 

GAO  09— Report  to  the  congressional  de¬ 
fense  committees  on  the  risks  and  mitiga¬ 
tion  strategy  for  use  of  cost  reimbursement 
contracts  for  procurement  and  plans  to 
transition  to  fixed  price  contracts 

o 

w 

w 

w 

c 

GAO  09— Ensure  contractor  performs  period¬ 
ic  schedule  risk  analyses  to  improve  schedule 
and  budget  actions 

o 

w 

w 

w 

c 

GAO  10— Make  a  new,  comprehensive,  and 
independent  assessment  of  the  costs  and 
schedule  to  complete  the  program,  including 
military  construction,  JSF-related  expenses 
in  other  budgets,  and  life-cycle  costs 

o 

w 

w 

c 

GAO  10— Reassess  warfighter  requirements 
and,  if  necessary,  defer  some  capabilities  to 
future  increments 

o 

w 

w 

c 

GAO  11— Maintain  future  funding  at  current 
levels 

o 

w 

c 

GAO  11— Establish  criteria  for  evaluating  the 
F-35B’s  progress  and  make  independent 
reviews,  allowing  each  variant  to  proceed  at 
its  own  pace 

o 

w 

c 

GAO  11— Conduct  an  independent  review  of 
the  software  development  and  lab  accredita¬ 
tion  processes 

o 

w 

c 

GAO  12-13— Restructure  JSF  program  by  in¬ 
corporating  previous  recommendations  from 
GAO  2008-11 

* 

GAO  14— Assess/identify  specific  capabilities 
that  can  be  delivered  to  the  military  services 
to  support  their  respective  initial  operational 
capabilities  by  July  2015 

o 

- 

GAO  14— Assess  the  affordability  of  F-35’s 
current  procurement  plan  that  reflects  var¬ 
ious  assumptions  about  technical  progress 
and  future  funding 

o 

- 

DoD  IG  13— Ensure  LM’s  design  and  material 
changes  are  with  government  concurrence 

o 

c 

DoD  IG  13— Perform  process  proofing  of 
supply  chain’s  critical  processes  and  require¬ 
ments  flow  verification 

o 

w 

w 

DoD  IG  13— Establish  independent  quality 
assurance  organization  to  review  supplier 
processes 

o 

w 

w 
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TABLE  2.  TRACKING  RECOMMENDATIONS  TO  THE  JPO,  CONTINUED 


Recommendations 

2005 

2006 

2007 

00 

o 

o 

N 

0) 

o 

o 

w 

o 

o 

N 

2011 

Ol 

O 

N 

2013 

2014 

2015 

DoD  IG  15— Evaluate  open  variance  on  F135 
engine 

o 

DoD  IG  15— Resolve  nonconformities  of 

P&W’s  engine  software  quality  management 
systems 

o 

Note.  Derived  and  summarized  from  DOT&E  (2005-14),  USGAO  (2005,  2007-14),  DoDIG 


(2013-15),  DOT&E  =  Director  of  Operational  Test  and  Evaluation,  GAO  =  Government 
Accountability  Office,  IG  =  Inspector  General,  PAO  =  Polyalphaolefin,  TEMP  =  Test  and 
Evaluation  Master  Plan,  IOT&E  =  Initial  Operational  Test  and  Evaluation,  SDD  =  system 
development  and  demonstration,  VSim  =  verification  simulator,  ALIS  =  Autonomic 
Logistics  Information  System,  OT  =  operational  test,  LO  =  low  observable,  msn  =  mission, 
fit  =  flight,  O  =  open,  W  =  working,  C  =  Close,  ?  =  unknown,  —  =  not  published  yet,  *  = 
summary 


Figure  2  shows  three  dense  reports'/journals*  co-occurrence/c-coefficient 
tables  generated  by  ATLAS.ti  QDA  software;  these  were  explained  in  depth 
by  the  dissertation  this  article  is  based  upon.  The  table  on  the  left  shows 
the  associations  between  acquisition  strategy  challenges  and  the  pillars  of 
acquisition.  The  middle  table  shows  the  interassociations  within  acquisi¬ 
tion  strategy  challenges.  The  table  on  the  right  shows  the  interassociations 
within  pillars  of  acquisition.  For  each  table,  each  coded  component  shows 
the  number  of  codes  in  parentheses  along  the  header  rows  and  columns.  At 
each  intersection,  the  co-occurrences  and  c-coefficients  are  paired.  The 
intent  here  is  to  show  some  specific  examples  to  eventually  discern  some 
obvious  general  trends.  Data  that  are  covered  up  by  overlapping  tables  are 
not  significant. 
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Note.  The  left  table  shows  strategies  versus  pillars.  The  middle  table  shows  strategies’ 
interassociations.  The  right  table  shows  pillars’  interassociations.  Data  that  are  covered  up 
by  overlapping  tables  are  not  significant. 

(n),  for  row/column  headers  =  number  of  occurrences  that  were  coded,  n/c  within  table 
where  n  =  co-occurrence  and  c  =  c-coefficient— bolded  represents  strong  association 
if  c  >  .08  and  at  least  two  co-occurrences,  “*”  represents  a  c-coefficient  that  may 
require  further  investigation— bolded  if  ratio  (r)  >  10  and  at  least  two  co-occurrences, 

BRI  =  Balancing  Requirements  Intent,  BRN  =  Balancing  Requirements  Negative,  BRP 
=  Balancing  Requirements  Positive,  BRR  =  Balancing  Requirements  Recommendation, 

Tl  =  Technology  Intent,  TN  =  Technology  Negative,  TP  =  Technology  Positive,  TR  = 
Technology  Recommendation,  CMI  =  Commonality  Intent,  CMN  =  Commonality  Negative, 
CMP  =  Commonality  Positive,  CMR  =  Commonality  Recommendation,  CCI  =  Concurrency 
Intent,  CCN  =  Concurrency  Negative,  CCP  =  Concurrency  Positive,  CCR  =  Concurrency 
Recommendation,  PTI  =  Partnering  Intent,  PTN  =  Partnering  Negative,  PTP  =  Partnering 
Positive,  PTR  =  Partnering  Recommendation 


The  left  table  in  Figure  2  has  acquisition  strategy  challenges'  component 
codes  depicted  in  the  left  column  and  the  pillars  of  acquisition  component 
codes  depicted  in  the  top  row.  Strong  associations  have  a  c-coefficient  > 
0.08  and  with  two  co-occurrences.  Circled  in  red,  out  of  the  27  co-occur¬ 
rences  between  TN  (Technology  Negative)  and  SLR  (Schedule  Lengthened 
Reason),  eight  co-occurrences  blamed  design  immaturity,  eight  co-occur¬ 
rences  blamed  the  lack  of  flight-testing  assets  that  lengthened  schedules,  six 
co-occurrences  blamed  complex  software,  and  three  co-occurrences  blamed 
weight  control.  In  the  middle  table,  acquisition  strategy  challenges  interas¬ 
sociations,  the  one  example  circled  in  green,  three  co-occurrences  between 
BRP  (Balancing  Requirements  Positive)  and  CMP  (Commonality  Positive) 
pointed  to  Joint  Services'  expected  savings  from  economies  of  scale.  In  the 
right  table,  pillars  of  acquisition  interassociations,  the  one  example  circled 
in  red,  out  of  the  15  co-occurrences  between  Cl  (Cost  Increase)  and  SL 
(Schedule  Lengthened),  12  co-occurrences  indicate  that  schedule  delays 
preceded  cost  growth,  but  six  out  of  the  12  pointed  to  technology  challenges 
being  the  root  cause. 

Figure  3  shows  percent  growth  or  decline  in  JSF  F-35  reported  program 
cost,  unit  cost,  and  acquisition  cycle  time  (official  program  start  to  full-rate 
production)  between  2001  and  2013  as  reported  by  the  General  Accounting 
Office  and  GAO.  The  percentage  represented  in  vertical  bars  is  based  on 
fiscal  year  (F  Y)  2015  dollars,  but  2001  is  the  base  year  shown  by  0%  change. 
The  2013  callout  box  is  also  in  F  Y  2015  dollars.  After  the  large  spike  in  unit 
cost  in  2003,  the  JSF  program  was  restructured,  but  the  program  triggered 
a  Nunn-McCurdy  breach  in  2007  for  exceeding  the  2003  restructured 
baseline  for  total  program  cost  and  again  in  2010  for  exceeding  the  2007 
restructured  baseline  (GAO,  2014).  From  2001  to  2013  in  FY  2015  dollars, 
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the  JSF  program's  total  program  cost  grew  49.7%,  its  program  unit  cost  grew 
74.7%,  and  the  acquisition  cycle  time  increased  104.3%,  doubling  the  time 
(General  Accounting  Office  2003, 2004;  Government  Accountability  Office 
2005-2011a,  2012a,  2013-2015). 


FIGURE  3.  JSF  PROGRAM  PERCENT  CHANGE  IN  COSTS,  QUANTITIES, 

AND  SCHEDULE 
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I  Research  and  Development  Cost  ■  Procurement  Cost  Total  Program  Cost 
I  Program  Unit  Cost  ■  Total  Quantities  ■  Acquisition  Cycle  Time 


Note.  Adapted  from  General  Accounting  Office  (2003,  2004)  and  Government 
Accountability  Office  (2005-2011a,  2012a,  2013-2015),  FY  =  fiscal  year 


Questionnaire  Responses  from  JSF  SMEs 

The  breakdown  of  questionnaire  respondents  was  slightly  different 
than  planned  because  potential  participants  were  coming  and  going  while 
the  author  was  coordinating  for  DoD  Information  Management  Control 
Office's  (IMCO)  approval  for  4  months.  Seventeen  questionnaires  were 
sent  to  the  managers  at  the  JPO,  25  to  the  customers  from  the  military  ser¬ 
vices,  and  eight  to  the  providers ,  the  contractors  at  LM.  If  there  had  been 
more  than  10  providers  (LM  contractors),  that  would  have  incurred  more 
IMCO  requirements  because  it  would  have  constituted  a  public  survey. 
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Each  participant  was  allowed  4  weeks  beginning  in  late  March  of  2015.  A 
50%  response  rate  was  expected  to  have  purposeful  sampling  according 
to  Zikmund,  Babin,  Carr,  and  Griffin  (2010)  and  Creswell  (2007).  Of  the 
50  potential  participants,  42  participants  responded  in  April  2015  for  an 
incredible  86%  response  rate— 10  out  of  17  from  the  JPO,  25  out  of  25  from 
the  Services,  and  seven  out  of  eight  from  LM  (see  Table  3).  However,  it  took 
5  weeks  for  the  seven  LM  responses  to  be  technically  reviewed  and  legally 
released  by  the  JPO  Security  Office  and  lawyers. 


TABLE  3.  PARTICIPANT  RESPONDENT  QUESTIONNAIRE  TIMING 


s. 

4-» 

■2 

Group 

Day  #1 

Completed 

Day  #28 

Released 

1 

1 

Manager 

19  Mar  15 

10  Apr  15 

16  Apr  15 

N/A 

5 

2 

Manager 

19  Mar  15 

9  Apr  15 

16  Apr  15 

N/A 

7 

3 

Manager 

19  Mar  15 

21  Apr  15* 

16  Apr  15 

N/A 

8 

4 

Manager 

19  Mar  15 

13  Apr  15 

16  Apr  15 

N/A 

10 

5 

Manager 

19  Mar  15 

14  Apr  15 

16  Apr  15 

N/A 

12 

6 

Manager 

19  Mar  15 

13  Apr  15 

16  Apr  15 

N/A 

14 

7 

Manager 

19  Mar  15 

13  Apr  15 

16  Apr  15 

N/A 

15 

8 

Manager 

19  Mar  15 

14  Apr  15 

16  Apr  15 

N/A 

16 

9 

Manager 

19  Mar  15 

15  Apr  15 

16  Apr  15 

N/A 

17 

10 

Manager 

19  Mar  15 

16  Apr  15 

16  Apr  15 

N/A 

18 

11 

Customer 

20  Mar  15 

10  Apr  15 

17  Apr  15 

N/A 

19 

12 

Customer 

20  Mar  15 

17  Apr  15 

17  Apr  15 

N/A 

20 

13 

Customer 

20  Mar  15 

17  Apr  15 

17  Apr  15 

N/A 

21 

14 

Customer 

20  Mar  15 

16  Apr  16 

17  Apr  15 

N/A 

22 

15 

Customer 

21  Mar  15 

20  Apr  15* 

18  Apr  15 

N/A 

23 

16 

Customer 

20  Mar  15 

3  Apr  15 

17  Apr  15 

N/A 

24 

17 

Customer 

20  Mar  15 

12  Apr  15 

17  Apr  15 

N/A 

25 

18 

Customer 

19  Mar  15 

14  Apr  15 

16  Apr  15 

N/A 

26 

19 

Customer 

21  Mar  15 

18  Apr  15 

18  Apr  15 

N/A 

27 

20 

Customer 

20  Mar  15 

6  Apr  15 

17  Apr  15 

N/A 
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TABLE  3.  PARTICIPANT  RESPONDENT  QUESTIONNAIRE  TIMING, 

CONTINUED 


4* 

c 

fO 

Q. 

u 


re 

0. 

i- 

Group 

Day  #1 

Completed 

Day  #28 

Released 

28 

21 

Customer 

14  Apr  15 

27  Apr  15 

12  May  15 

N/A 

29 

22 

Customer 

19  Mar  15 

9  Apr  15 

16  Apr  15 

N/A 

30 

23 

Customer 

23  Mar  15 

10  Apr  15 

20  Apr  15 

N/A 

31 

24 

Customer 

20  Mar  15 

17  Apr  15 

17  Apr  15 

N/A 

32 

25 

Customer 

20  Mar  15 

16  Apr  15 

17  Apr  15 

N/A 

33 

26 

Customer 

20  Mar  15 

16  Apr  15 

17  Apr  15 

N/A 

34 

27 

Customer 

21  Mar  15 

14  Apr  15 

18  Apr  15 

N/A 

35 

28 

Customer 

31  Mar  15 

27  Apr  15 

28  Apr  15 

N/A 

36 

29 

Customer 

21  Mar  15 

14  Apr  15 

18  Apr  15 

N/A 

37 

30 

Customer 

21  Mar  15 

7  Apr  15 

18  Apr  15 

N/A 

38 

31 

Customer 

21  Mar  15 

9  Apr  15 

18  Apr  15 

N/A 

39 

32 

Customer 

21  Mar  15 

17  Apr  15 

18  Apr  15 

N/A 

40 

33 

Customer 

22  Mar  15 

19  Apr  15 

19  Apr  15 

N/A 

41 

34 

Customer 

21  Mar  15 

18  Apr  15 

18  Apr  15 

N/A 

42 

35 

Customer 

26  Mar  15 

30  Mar  15 

23  Apr  15 

N/A 

43 

36 

Provider 

19  Mar  15 

15  Apr  15 

16  Apr  15 

26  May  15 

44 

37 

Provider 

19  Mar  15 

15  Apr  15 

16  Apr  15 

26  May  15 

46 

38 

Provider 

19  Mar  15 

7  Apr  15 

16  Apr  15 

26  May  15 

47 

39 

Provider 

19  Mar  15 

16  Apr  15 

16  Apr  15 

26  May  15 

48 

40 

Provider 

19  Mar  15 

8  Apr  15 

16  Apr  15 

26  May  15 

49 

41 

Provider 

19  Mar  15 

8  Apr  15 

16  Apr  15 

26  May  15 

50 

42 

Provider 

19  Mar  15 

3  Apr  15 

16  Apr  15 

26  May  15 

Note.  *Late  submission  allowed  by  Dissertation  Chair 
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Table  4  speaks  to  the  credibility  of  the  respondents.  From  the  top  of  Table 
4,  although  there  were  only  seven  LM  contractor  (“Con”)  respondents  due 
to  not  making  the  questionnaire  a  public  survey,  active  duty  (AD)  and  gen¬ 
eral  schedule  (GS)  civilian  respondents  were  practically  even  at  17  and  18, 
respectively.  A  majority  (34  of  42  or  81%)  of  the  respondents  had  earned 
a  master's  degree  as  their  highest  level  of  education.  Most  of  the  respon¬ 
dents  were  either  executives/directors,  program  managers  (of  the  variant 
or  subsystem),  or  action  officers  with  a  few  dedicated  pilots,  and  a  tester.  A 
strong  majority  (69%)  of  the  respondents  considered  themselves  SMEs,  a 
third  of  the  respondents  had  pilot  experience,  a  few  considered  themselves 
as  maintainers  (12%),  and  several  (19%)  as  policy  deciders.  By  far,  the 
providers  from  LM  had  the  most  years  of  experience  in  dealing  with  each 
acquisition  strategy— 40%  more  than  the  managers  at  the  JPO,  and  more 
than  double  the  customers  from  the  Services.  Overall,  respondents  had  the 
most  years  of  experience  when  dealing  with  Harnessing  Technology,  fol¬ 
lowed  by  Balancing  Requirements,  followed  by  a  near  three-way  tie  between 
the  remaining  strategy  challenges.  The  large  standard  deviations  in  years 
of  experience  was  because  several  respondents  (12  of  42,  29%)  had  zero 
experience  with  more  than  one  associated  acquisition  strategy  challenge, 
but  they  were  all  highly  recommended  by  their  superiors  to  be  invited  to 
participate  in  the  study. 
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TABLE  4.  BREAKDOWN  OF  RESPONDENTS’  DEMOGRAPHICS 


January  2017 


Tot 

o 

25 

is 

42 

X 

Q 

o 

Q 

CM 

4-» 

C 

00 

|S 

G) 

K> 

Z 

CM 

<D 

LO 

s 

cb 

■o 

, 

, 

£ 

O 

ro 

C 

0 

1 

+ 

+ 

■ 

s 

+ 

O 

a 

(A 

o 

K) 

CM 

in 

MS 

00 

CM 

LO 

34 

Pilot 

O 

00 

33 

0 

a 

O 

cb 

CO 

BS 

- 

CM 

- 

SME 

o 

CO 

O) 

u\ 

c 

s 

K) 

is 

O 

IS 

cb 

HS 

o 

o 

i— 

rs 

CD 

00 

(0 

<5 

c 

H 

0 

a 

V 

00 

i 

+ 

K) 

i 

+ 

03 

■ 

s 

+ 

o 

00 

o 

cb 

0 

D 

O 

CO 

CD 

0) 

c 

s, 

K) 

CM 

0 

(0 

o 

3 

■o 

111 

JPO 

iz 

LM 

Total 

o 

a 

c 

.2  £ 
c  < 

JPO 

Z 

LM 

OS 

"fo 

4-» 

o 

> 

o 

c 

2 

k. 

3 

U 

c 

K) 

LO 

1 

+ 

K) 

LO 

i 

+ 

rs 

.1  +/-  5.3 

LO 

■ 

s 

+ 

3  ^ 

£ 

0 

cb 

LO 

CD 

cb 

42 

1 L 

u 

£ 

o 

LO 

CM 

rs 

LO 

CM 

CM 

>1 

C 

£ 

o 

IS 

•M 

K) 

IS 

00 

o 

o 

O 

|S 

is 

fo 

cb 

'st 

O) 

cb 

o 

c 

i 

■ 

0 

\ 

E 

+ 

+ 

+ 

+ 

S9 

LO 

K) 

O 

00 

C 

k 

< 

o 

LO 

o 

in 

E 

0 

LO 

cb 

CM 

cb 

a 

o 

AD 

LO 

CM 

O 

Is 

O 

k 

>1 

CD 

00 

o 

id 

o 

0) 

o 

j-i 

.. 

IA 

o 

o 

f— 

0 

CD 

»— 

o 

i 

Q. 

£ 

JPO 

Z 

LM 

Total 

£ 

0 

c 

S 

£ 

1 

+ 

is 

+ 

LO 

(7) 

V 

o 

o 

CM 

\ 

+ 

CM 

to 

c 

111 

AO 

o 

LO 

o 

in 

(A 

Tot 

O 

LO 

CM 

IS 

42 

** 

C 

0 

E 

|S 

s 

LO 

O) 

0> 

PM 

1 

3 

i 

\ 

Z 

o 

o 

|S 

IS 

CM 

O 

\ 

+ 

+ 

00 

+ 

O 

+ 

-i 

O' 

0 

rs 

s 

o 

a 

LO 

in 

z 

o 

CM 

o 

CM 

O  i. 

s  5 

LO 

o 

LO 

0  O 

"0 

0> 

'C 

0> 

c: 

's 

0> 

■>> 

o 

iii 

u  ** 

0) 

a 

o 

O 

o 

O 

ia  :W 

■i-  .2 
ib  "> 

03 

r~ 

o 

„Cj 

— 

| 

4-j 

Co 

2 

0 

4-» 

Org: 

JPO 

z 

LM 

Total 

Job 

(One) 

JPO 

z 

LM 

Total  #; 

p  0  + 

*  &§* 
iii  4 
w 

o 

CL 
~ ) 

^3 

o 

z 

a 

z 

_i 

£ 

Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


131 


Note.  JPO  =  JSF  Program  Office,  Mil  =  military  services,  LM  =  Lockheed  Martin,  Tot  =  total,  AD  =  active  duty,  GS  =  general  schedule,  Con  = 
contractor,  HS  =  high  school,  BS  =  bachelor’s,  MS  =  master’s,  Dctr  =  doctorate,  Exec  =  executive,  Dir  =  director,  PM  =  program  manager,  AO 
=  action  officer,  Op  =  operator,  Trn  =  Trainer,  Pol  Dec  =  policy  decider,  SME  =  subject  matter  expert,  Mx  =  maintained  Avg  =  average,  StD  = 
standard  deviation 
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Similar  to  “years'  experience"  from  the  bottom  of  Table  4,  Table  5  shows  the 
average  amount  of  words  respondents  used  to  answer  the  questions.  The 
customers  had  the  most  to  say  at  1,520  words  per  an  entire  set  of  answers, 
almost  70%  more  than  the  managers,  and  almost  twice  as  much  as  the  pro¬ 
viders.  This  order  held  true  for  the  number  of  words  per  each  acquisition 
strategy  question  except  for  commonality,  where  the  managers  and  provid¬ 
ers  switched  places  for  second  and  third. 

Respondents  had  the  most  to  say  about  Balancing  Requirements,  with 
an  average  of  114  words  for  that  section  of  three  questions,  followed  by 
Harnessing  Technology  and  Encouraging  Partnering,  with  89  and  83  words 
per  section,  respectively,  followed  by  Evoking  Concurrency  and  Demanding 
Commonality,  with  75  and  55  words,  respectively.  The  standard  deviations 
were  also  large,  but  the  quality  of  answers  should  not  be  attributed  to  the 
number  of  words. 
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TABLE  5.  RESPONDENTS’  WORD  QUANTITIES 


January  2017 
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Table  6  shows  the  breakdown  of  family  codes  by  the  type  of  respon¬ 
dent-manager  (JPO),  customer  (Services),  or  provider  (LM).  Out  of  1,564 
respondent  codes,  customers  from  the  Services  provided  the  most  codes,  but 
they  were  proportional  to  being  the  largest  group  of  respondents.  Customer 
respondents'  codes  were  about  2.5  times  more  than  the  managers  from  the 
JPO  and  four  times  more  than  the  providers  from  LM.  These  proportions 
held  throughout  the  acquisition  strategy  challenges'  codes,  indicating 
consistency  in  coding.  Although  the  proportional  relationship  of  total 
coding  versus  the  family  codes  of  cost,  schedule,  and  performance  did  not 
stand  up  as  well,  the  coding  proportions  within  cost,  schedule,  and  per¬ 
formance  were  consistent.  However,  for  all  coding,  remaining  objective 
was  challenging  for  the  researcher  and  required  an  iterative  process  of 
recoding  to  ensure  that  the  respondents  were  actually  commenting  about 
an  acquisition  strategy  challenge  in  its  relation  to  one  or  more  of  the  pillars 
of  acquisition.  It  is  interesting  to  point  out  that  Balancing  Requirements 
and  Harnessing  Technology  had  the  most  negative  codes,  with  91  and  89. 
However,  Balancing  Requirements  and  Demanding  Commonality  had  the 
most  positive  codes,  with  62  and  56.  It  is  interesting  that  all  the  acquisition 
strategy  challenges  have  nearly  the  same  number  of  recommendations 
coded— about  50.  Within  the  pillars,  the  positive  categories  of  Cost  Decrease 
and  Cost  Decrease  Reason  were  coded  the  most  with  79  and  77  codes.  On 
the  negative  side,  Performance  Reduced  and  Cost  Increase  Reason  had  the 
highest  codes  with  68  and  52. 
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TABLE  6.  RESPONDENTS’  FAMILY  CODE  BREAKDOWN 

Manager 

Program  Office 

(10) 

Customer 

Services 

(25) 

Provider 

Contractors 

(7) 

Totals 

(42) 

Balancing  Requirements  Intent  (BRI) 

13 

37 

12 

62 

Balancing  Requirements  Negative  (BRN) 

21 

54 

16 

91 

Balancing  Requirements  Positive  (BRP) 

7 

26 

11 

44 

Balancing  Requirements 

Recommendation  (BRR) 

9 

30 

8 

47 

Balancing  Requirements  Sub-total 

49 

144 

43 

236 

Technology  Intent  (Tl) 

9 

23 

7 

39 

Technology  Negative  (TN) 

15 

61 

13 

89 

Technology  Positive  (TP) 

3 

21 

7 

31 

Technology  Recommendation  (TR) 

15 

30 

5 

50 

Harnessing  Technology  Sub-total 

40 

134 

30 

204 

Commonality  Intent  (CMI) 

14 

28 

14 

56 

Commonality  Negative  (CMN) 

7 

39 

3 

49 

Commonality  Positive  (CMP) 

19 

32 

12 

63 

Commonality  Recommendation  (CMR) 

9 

24 

4 

37 

Demanding  Commonality  Sub-total 

41 

118 

30 

204 

Concurrency  Intent  (CCI) 

13 

31 

6 

50 

Concurrency  Negative  (CCN) 

16 

46 

7 

69 

Concurrency  Positive  (CCP) 

9 

28 

8 

45 

Concurrency  Recommendation  (CCR) 

13 

31 

6 

50 

Evoking  Concurrency  Sub-total 

49 

133 

26 

189 

Partnering  Intent  (PTI) 

12 

33 

5 

50 

Partnering  Negative  (PTN) 

10 

42 

4 

56 

Partnering  Positive  (PTP) 

14 

40 

9 

63 

Partnering  Recommendation  (PTR) 

7 

32 

4 

43 

Encouraging  Partnering  Sub-total 

41 

143 

20 

204 

Cost  Increase  (Cl)— bad 

16 

23 

6 

45 

Cost  Increase  Reason  (CIR)— why 

22 

22 

8 

52 

Cost  Decrease  (CD)— good 

14 

44 

21 

79 

Defense ARJ,  January  2017,  Vol.  24 No.  1 : 102-160 


135 


Scandal  and  Tragedy? 


http://www.dau.mil 


TABLE  6.  RESPONDENTS’  FAMILY  CODE  BREAKDOWN, 

,  CONTINUED 

Manager 

Program  Office 

(10) 

Customer 

Services 

(25) 

Provider 

Contractors 

(7) 

Totals 

(42) 

Cost  Decrease  Reason  (CDR)— why 

32 

32 

13 

77 

Cost  Recommendation  (CR) 

2 

2 

0 

4 

Cost  Sub-total 

86 

120 

48 

254 

Schedule  Lengthened  (SL)— bad 

16 

25 

10 

51 

Schedule  Lengthened  Reason  (SLR)— why 

13 

21 

7 

41 

Schedule  Shortened  (SS)— good 

8 

19 

7 

34 

Schedule  Shortened  Reason  (SSR)— why 

4 

4 

0 

8 

Schedule  Recommendation  (SR) 

3 

0 

1 

4 

Schedule  Sub-total 

43 

69 

25 

137 

Performance  Reduced  (PRD)— bad 

14 

50 

4 

68 

Performance  Reduced  Reason  (PRR)— why 

8 

19 

3 

30 

Performance  Improved  (PI)— good 

4 

13 

3 

20 

Performance  Increased  Reason  (PIR) — why 

3 

5 

1 

9 

Performance  Recommendation  (PR) 

2 

5 

0 

7 

Performance  Sub-total 

31 

90 

11 

132 

Grand  Totals 

380 

951 

233 

1564 

Note.  (XX)  =  number  of  respondents 

Similar  to  Figure  2,  Figure  4  shows  three  questionnaire  respondents' 
co-occurrences/c-coefficient  tables  generated  by  ATLAS.ti  QDA  software; 
these  were  explained  in  depth  by  the  dissertation  this  article  is  based  upon. 
The  table  on  the  left  shows  the  associations  between  acquisition  strategy 
challenges  and  the  pillars  of  acquisition.  The  middle  table  shows  the  inter¬ 
associations  within  acquisition  strategy  challenges.  The  table  on  the  right 
shows  the  interassociations  within  the  pillars  of  acquisition.  For  each  table, 
each  coded  component  shows  the  number  of  codes  in  parentheses  along  the 
header  rows  and  columns.  At  each  intersection,  the  co-occurrences  and 
c-coefficients  are  paired.  The  intent  here  is  to  show  some  specific  examples 
to  eventually  discern  some  obvious  general  trends.  Data  that  are  covered  up 
by  overlapping  tables  are  not  significant. 
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FIGURE  4.  RESPONDENTS’  CO-OCCURRENCE/C-COEFFICIENT  TABLES 
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Note.  The  left  table  shows  strategies  versus  pillars.  The  middle  table  shows  strategies’ 
interassociations.  The  right  table  shows  pillars’  interassociations.  Data  that  are  covered  up 
by  overlapping  tables  are  not  significant. 

(n),  for  row/column  headers  =  number  of  occurrences  that  were  coded,  n/c  within  table 
where  n  =  co-occurrence  and  c  =  c-coefficient— bolded  represents  strong  association 
if  c  >  .08  and  at  least  two  co-occurrences,  “*”  represents  a  c-coefficient  that  may 
require  further  investigation— bolded  if  ratio  (r)  >  10  and  at  least  two  co-occurrences, 

BRI  =  Balancing  Requirements  Intent,  BRN  =  Balancing  Requirements  Negative,  BRP 
=  Balancing  Requirements  Positive,  BRR  =  Balancing  Requirements  Recommendation, 

Tl  =  Technology  Intent,  TN  =  Technology  Negative,  TP  =  Technology  Positive,  TR  = 
Technology  Recommendation,  CMI  =  Commonality  Intent,  CMN  =  Commonality  Negative, 
CMP  =  Commonality  Positive,  CMR  =  Commonality  Recommendation,  CCI  =  Concurrency 
Intent,  CCN  =  Concurrency  Negative,  CCP  =  Concurrency  Positive,  CCR  =  Concurrency 
Recommendation,  PTI  =  Partnering  Intent,  PTN  =  Partnering  Negative,  PTP  =  Partnering 
Positive,  PTR  =  Partnering  Recommendation 


The  table  on  the  left  of  Figure  4  has  acquisition  strategy  challenges'  com¬ 
ponent  codes  depicted  in  the  left  column  and  the  pillars  of  acquisition 
component  codes  depicted  in  the  top  row.  There  were  several  more  strong 
associations  circled  for  this  analysis  lane  as  compared  to  the  Reports'/ 
Journals'  lane,  and  each  strong  association  was  explained  in  depth  in  the 
dissertation  this  article  is  based  upon.  For  the  green  circle  in  the  mid¬ 
dle-left— 14  of  the  21  co-occurrences  between  CMP  (Commonality  Positive) 
and  CD  (Cost  Decreased)  believed  there  has  been  and  will  continue  to  be 
cost  savings.  For  the  red  circle  just  below,  all  16  co-occurrences  between 
CCN  (Concurrency  Negative)  and  CIR  (Cost  Increase  Reason)  understood 
that  all  the  early  jets,  around  a  100,  need  significant  and  costly  modifica¬ 
tions  that  exceeded  expectations.  For  the  orange  circle  to  the  right  between 
CCR  (Concurrency  Recommendation)  and  CR  (Cost  Recommendation), 
the  c-coefficient  required  further  investigation  and  yielded  one  of  the  best 
recommendations  from  a  manager,  “Maintain  program  discipline  up  front, 
don't  over-promise"  in  the  ability  to  maintain  planned  concurrency.  For 
the  second  green  circle  at  the  bottom,  14  of  the  18  co-occurrences  between 
PTP  (Partnering  Positive)  and  CDR  (Cost  Decrease  Reason)  reiterated  the 
savings  of  having  multiple  countries  investing  in  development  and  reducing 
the  price  per  unit.  The  remaining  four  occurrences  pointed  out  that  Partner 
countries  were  given  economic  opportunities  to  produce  or  maintain  sig¬ 
nificant  aspects  of  the  F-35. 

In  the  middle  table,  acquisition  strategy  challenges  interassociations,  for 
the  red  circle,  8  of  the  12  co-occurrences  between  BRN  (Balancing 
Requirements  Negative)  and  CMN  (Commonality  Negative)  emphasized 
the  resulting  differences  and  uncompromising  tradeoffs  between  the  F-35 
variants  in  terms  of  range  requirements,  fuel  capacities,  and  weights.  In  the 
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right  table,  pillars  of  acquisition  interassociations,  for  the  green  circle,  the 
19  co-occurrences  between  CD  (Cost  Decreased)  and  SS  (Schedule 
Shortened)  read  more  like  revisionist  history  that  has  not  occurred  yet— 
seven  of  19  co-occurrences  praised  commonality  based  on  future 
expectations  that  commonly  designed  parts  will  pay  off  huge  dividends  in 
lower  sustainment  costs  for  40  more  years  of  the  program  and  that  common 
mission  systems  (mission  planning  and  sensors)  will  reduce  the  need  for 
Services  and  other  partner  nations  to  look  for  expensive  alternatives. 
Another  seven  co-occurrences  opined  that  balancing  requirements  would 
have  produced  earlier  savings  if  no  changes  in  requirements  were  made. 
Although  three  co-occurrences  believed  that  concurrency  is  currently  suc¬ 
cessful  in  reducing  cost  and  time,  its  current  reputation  is  quite  the  opposite. 
For  the  red  circle,  out  of  the  31  co-occurrences  between  Cl  (Cost  Increase) 
and  SL  (Schedule  Lengthened),  12  were  attributed  to  immature  technology, 
10  to  balancing  requirements,  three  to  commonality,  four  to  concurrency, 
and  two  to  partnering— all  five  of  the  acquisition  strategy  challenges  the 
2015  study  focused  on. 


... acquisition  strategies  have  both  aided  and 
hindered  joint  aircraft  programs’  schedule,  cost,  and 
performance  in  terms  of  Balancing  Requirements, 
Harnessing  Technology,  Demanding  Commonality, 
Evoking  Concurrency,  and  Encouraging  Partnering. 


Answering  the  Research  Questions 

In  answering  the  main  research  question,  the  2015  study  helped  us 
understand  to  a  deeper  and  richer  level  that  acquisition  strategies  have  both 
aided  and  hindered  joint  aircraft  programs'  schedule,  cost,  and  performance 
in  terms  of  Balancing  Requirements,  Harnessing  Technology,  Demanding 
Commonality,  Evoking  Concurrency,  and  Encouraging  Partnering.  The 
following  sections  will  answer  each  subquestion  (SQ)  about  the  acquisi¬ 
tion  strategy's  original  intent,  its  positive  and  negative  effects  on  schedule, 
cost,  and  performance,  and  what  improvements  could  be  made.  Individual 
acquisition  strategy  challenge  taxonomies  will  show  relationships  between 
strategies  and  pillars,  and  emphasize  relativeness  of  what  was  learned  from 
each  analysis  lane  from  the  2015  study,  and  together  will  help  summarize 
lessons  learned. 
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For  each  acquisition  strategy  challenge's  summary  taxonomy,  there  will  be 
contributing  factors  and  arrows  in  the  appropriate  color  matching  to  the 
previous  synthesized  Figure  1  or  to  the  previous  co-occurrence/c-coefficient 
tables  in  Figures  2  and  4.  The  previous  joint  aircraft  programs'  analysis 
lane  will  be  depicted  in  brown,  government  and  think  tank  reports  and 
scholarly  journals  will  be  in  purple,  and  respondents'  questionnaires  will 
be  in  black.  The  components  for  acquisition  strategy  challenges  and  pillars 
of  acquisition  will  usually  be  in  red/  negative  or  green/positive  boxes  if  they 
emanated  from  a  strong  association  starting  with  an  acquisition  strategy 
challenge.  Otherwise,  some  component  boxes  will  remain  clear.  Within  the 
tan  circle  of  acquisition  strategy  challenges,  for  strong  relationships  to  other 
strategies,  only  contributing  factors  from  the  analysis  lane  from  which  they 
came  from  will  be  shown  in  the  appropriate  colored  text.  Within  the  olive 
area,  contributing  factors  between  pillars  of  acquisition  components  will 
be  shown  in  white  text  over  the  appropriate  colored  arrow  to  also  indicate 
from  which  analysis  lane  it  came. 


SQ1  on  Balancing  Requirements 

The  original  intent  ofbalancing  requirements,  especially  with  joint  air¬ 
craft  programs,  is  to  merge  and  trade  off  the  needs  of  several  U.S.  military 
services  and  nations  into  one  weapon  system,  thus  combining  several  pro¬ 
grams  into  one  in  order  to  save  money  and  time  as  opposed  to  each  Service 
or  country  developing  its  own  aircraft  program.  The  result  of  Balancing 
Requirements  would  be  a  weapon  system  that  has  commonality  in  design, 
structure,  mission  systems,  and  parts  among  possible  variants.  Specifically, 
the  JSF  program  initially  combined  eight  U.S.  military  services'  programs: 
Advanced  Short  Take-Off/Vertical  Landing,  the  Short  Take-Off/Vertical 
Landing  Strike  Fighter,  the  Common  Affordable  Lightweight  Fighter,  the 
Multi-Role  Fighter,  the  Advanced  Tactical  Aircraft,  the  Naval  Advanced 
Tactical  Fighter,  the  Advanced  Attack/Advanced/Fighter  Attack,  and  the 
Joint  Advanced  Strike  Technology  programs  (Antill  &  Ito,  2013);  and  then 
the  JSF  program  became  the  leading  fighter  replacement  program  candidate 
for  many  foreign  nations. 


Balancing  capability  requirements  can  have  a  positive  effect 
on  cost  by  combining  several  programs  into  one  and  by 
getting  years  of  sustainment  savings  due  to  common 
parts  over  a  very  large  pool  of  aircraft.  In  some 

respect,  Balancing 
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Requirements  reduces  schedule  because  of  the  one  single  design  effort 
and  the  consolidation  of  test  points  between  variants.  On  the  other  hand, 
balancing  capability  requirements  has  had  a  negative  effect  on  the  JSF  sched¬ 
ule  because  it  has  taken  longer  than  expected  for  the  Services  to  agree  on 
requirements  since  they  operate  in  different  environments.  Although  com¬ 
monality  is  great  in  the  long  term  for  parts  replacement,  in  the  short  term, 
the  JSF  program  took  more  time  and  money  to  design  those  common  parts 
within  specified  weight  criteria,  especially  to  meet  the  requirements  of  the 
USMC's  F-35B,  which  took  a  SWAT  (STOVL  Weight  Attack  Team)  to  solve. 
During  the  time  it  took  for  the  Services  to  compromise  and  the  engineers  to 
design,  Services  updated  and  added  key  performance  parameters  (KPP)  as 
the  threat  evolved,  thereby  increasing  cost. 

To  improve  the  process  of  balancing  capability  requirements,  one  can 
either  force  performance-affordability  compromises  between  the  Services 
earlier  and  more  often  or  be  willing  to  diverge  from  the  designs  between 
the  variants  earlier.  As  we  learned  from  questionnaire  respondents  only, 
initial  airframe  commonality  expectation  was  80%,  and  this  was  reduced 
in  actuality  to  30  to  40%  by  2015.  However,  respondents  highlighted  that 
mission  systems  and  avionics  between  all  variants  were  between  95  and 
100%.  The  USMC  gave  up  time,  but  still  got  their  STOVL  replacement  to 
the  AV-8  Harrier.  The  USMC  did  not  care  as  much  about  low  observability 
as  the  USAF  did,  so  much  so  that  the  USMC  ditched  the  internal  gun  for 
an  external  gun  pod  that  increases  the  radar  cross  section.  Although  the 
USAF's  desire  for  an  internal  gun  conflicted  directly  with  the  USMC's  need 
for  a  vertical  lift  fan  in  almost  the  same  location  of  the  aircraft,  the  USAF 
did  lose  speed  and  range  performance  to  incorporate  other  aspects  that  were 
needed  for  USMC  STOVL  engine  capability. 

As  a  matter  of  relativeness  between  the  analysis  lanes,  Figure  5  shows  a 
summary  taxonomy  of  all  the  Balancing  Requirements  components'  con¬ 
tributing  factors  and  how  they  affect  the  pillars  of  acquisition  and  how 
the  pillars  of  acquisition  react  to  one  another  under  the  guise  of  balancing 
requirements.  We  learned  from  previous  joint  aircraft  programs  (in  brown), 
especially  from  the  1960s  TFX  F-lll,  that  combining  multiple  Services' 
requirements  is  a  difficult  challenge.  The  difference  with  the  JSF  program 
is  that  the  USMC  needs  the  F-35B  more  now  than  the  USN  needed  the 
F-lll  back  then.  Numerous  negative  contributing  factors,  mostly  from  the 
previous  joint  aircraft  program  analysis  (in  brown)  and  from  the  respon¬ 
dent  questionnaire  analysis  (in  black),  are  more  prevalent  than  positive 
contributing  factors,  especially  in  relation  to  cost,  which  RAND  concluded 
cost  more  for  joint  programs  than  for  multiple  single-Service  programs 
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(Lorell  et  al.,  2013).  The  relationship  that  stands  out  within  the  pillars  of 
acquisition  is  that  overpromised  performance,  coupled  by  underestimated 
cost  and  schedule,  resulted  in  Services'  capability  tradeoffs. 


FIGURE  5.  BALANCING  REQUIREMENTS  SUMMARY  TAXONOMY 


Combine  Programs  to  Get  One  Aircraft 
Jointness  Trade-offs 
One  Program  Avoiding  Three 
Compatible  Variants 
Means  to  an  Ends  of  Commonality 

Invest  Into  One  Aircraft  to  Meet" 
Multiservice  Needs 
Meet  Timeline  Milestones  for  All  Involved 
Combining  Test  Points 
Meet  Numerous  U.S.  Service  &  Foreign 
Countries’  Needs 
Combining  Test  Points 
Meet  Numerous  U.S.  Service  &  Foreign 
Countries’  Needs^ 

Different  Services’  Operating  Environments"' 
Services’  Changing  Requirements 
Driving  Consensus  Forces  Concessions 
Increased  Weight 
Joint  Programs  More  Expensive 
Meeting  Each  Service’s  Needs 
F-35B’s  STOVL  Weight,  F-35A’s  Internal  Gun 
Services’  Unwillingness  to  Compromise 
Compromise  Delay  Allows  Threats  to 
Counter 

Reduced,  Compromised,  or  Traded-off  KPPs 

Force  Performance-Affordability  Compromises 
Performance-Affordability  Compromises 
Divergence  Positives 


One  Aircraft  to  Meet  Needs  of  Two  or  More 
Services 

Economies  of  Scale 
Jointness  Trade-offs 
Sharing  Part  Reduce  Life-cycle  Costs 
Affordable  &  Interchangeable 
Economies  of  Scale  of  3  Programs  in  1 
Development/Sustainment  Efficiencies 
One-solution,  Interoperable  Multi-variant 
Smaller  Logistics,  Simplified  Supportability 


Economies  of  Scale 

More  Production  Orders,  Larger  Parts 

Pool 

Sharing  Part  Reduce  Life-cycle  Costs 


Services’  Contradictory  Aerodynamic 
Requirements 

%  of  Common  Parts  Decreases  with  Time 
More  Stakeholders  Take  More  Time 
Threatens  Service  Autonomy 
Divergence  Amongst  Variants 
USN  F-35C  Environment  Challenges 
Untested  Integration 
Sacrifice  in  Capabilities 
Compromising  Takes  Time 
F-35B  Weight  Requirements 


Force  Performance-Affordability  Compromises 


Note.  Cl  =  Cost  Increased,  CIR  =  Cost  Increase  Reason,  CD  =  Cost  Decreases,  CDR  =  Cost 
Decrease  Reason,  CR  =  Cost  Recommendation,  SL  =  Schedule  Lengthened,  SS  =  Schedule 
Shortened,  SSR  =  Schedule  Shortened  Reason,  SR  =  Schedule  Recommendation,  PRD  = 
Performance  Reduced,  PRR  =  Performance  Reduced  Reason,  PI  =  Performance  Improved, 
BRI  =  Balancing  Requirements  Intent,  BRN  =  Balancing  Requirements  Negative,  BRP  = 
Balancing  Requirements  Positive,  BRR  =  Balancing  Requirements  Recommendation,  CMI  = 
Commonality  Intent,  CMN  =  Commonality  Negative,  CMP  =  Commonality  Positive,  CMR  = 
Commonality  Recommendation 


Red  =  Previous  Joint  Aircraft  Programs,  Blue  =  Government/Think  Tank  Reports  and 
Scholarly  Journals,  Black  =  Respondents’  Questionnaires 


SQ2  on  Harnessing  Technology 

The  original  intent  of  harnessing  technology  is  for  the  weapon  system 
to  give  warfighters  an  asymmetric  advantage  over  their  potential  adver¬ 
saries.  Although  potential  adversaries  have  developed  low-observable  (LO) 
cruise  missiles,  the  United  States  is  considered  the  leader  in  aircraft  LO 
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development.  That  development  started  with  the  F-117  stealth  fighter  and 
continues  with  the  B-2  stealth  bomber  and  the  F-22  fighter— which  are  being 
slowly  countered  by  potential  adversaries.  As  reported  in  annual  DOT&E 
reports  (2004-2014),  the  F-35  fighter  is  expected  to  be  the  most  advanced 
and  dominant  aircraft  in  the  world  for  decades  to  come,  not  just  for  its  next 
generation  LO  stealth  technology,  but  for  its  fused  and  integrated  sensors, 
weapons,  and  electronic  attack  used  in  conjunction  with  LO. 

Harnessing  technology  is  not  cheap,  and  it  takes  time,  but  it  eventually 
improves  tactical  performance.  Even  with  COTS  technology,  the  military 
services  eventually  manipulate  the  technology  to  meet  military  specifica¬ 
tions  much  like  what  happened  to  the  1990s  JPATS  T-6.  In  conjunction  with 
concurrency,  numerous  advanced  technologies,  even  if  somewhat  immature, 
can  be  brought  to  the  warfighter  quicker  by  flying  with  the  systems  sooner 
to  fix  problems  earlier  so  aircrews  can  have  the  chance  to  survive  against 
an  advanced  threat.  What  makes  the  F-35  and  the  F-22  “5th  Gen”  is  not 
just  its  low  observability,  but  its  fusion  of  sensors.  The  F-35  uses  LO  stealth 
structure,  active-electronically  scanned  array  (AESA)  radar,  and  electronic 
warfare  that  has  incrementally  improved  over  the  last  20  years.  Although 
air  forces  have  used  helmets  with  displays  in  the  visor  for  over  15  years, 
the  JSF's  helmet  mounted  display  system  (HMDS)  is  so  advanced  that  you 
cannot  employ  the  F-35  without  it,  and  if  it  fails,  the  pilot  returns  to  base. 
The  latest  and  greatest  technologies  are  the  electro-optical  targeting  sys¬ 
tem  (EOTS;  forward  looking  air-to-air  and  air-to-ground  infrared  search 
and  track  system)  and  the  distributed  aperture  system  (a  360-degree  day/ 
night  electro-optical  system),  which  give  the  pilot  a  protective  sphere  of 
situational  awareness  of  incoming  aircraft  and  missile  threats. 

Making  sure  these  technologies  work  as  advertised  usually  takes  more 
time  and  money  than  planned.  Much  like  the  1960s  TFX  F-lll  variable 
wing  sweep  (for  high  and  low  speeds)  and  variable  inlet  (for  the  new  engine) 
and  the  1980s  JVX  V-22  tilt  rotor  technology  challenges,  the  JSF  F-35 
experienced  many  technology  challenges  just  to  fit  the  unique  lift  fan 
engine  design  for  the  USMC  F-35B  STOVL  into  the  same  basic  airframe 
mid-section  for  all  F-35  variants  (DOT&E,  2010)— part  of  the  negativeness 
of  commonality.  Not  only  was  the  F-35B  overweight  by  several  hundred 
pounds  for  quite  some  time  before  a  solution  was  found  (Blickstein  et  al., 
2011),  but  the  HMDS,  which  is  exactly  the  same  for  all  variants,  has  also 
been  too  heavy  by  ounces  to  be  safe  during  ejection,  and  had  a  jittery  display 
that  was  fixed  separately  (GAO,  2014).  Concerning  academics  courseware 
for  pilot  and  maintenance  training  and  for  aircraft  maintenance  tracking 
records,  ALIS  is  still  immature  and  requires  workarounds  that  use  legacy 
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academic  courseware  and  legacy  maintenance-tracking  systems  (Everstine, 
2015).  The  root  causes  to  all  these  late  schedules  and  cost  increases  are  due 
to  the  complex  and  immature  software  that  is  still  being  developed,  written, 
and  tested.  Many  of  the  requirements'  developers  within  each  Service  were 
overly  optimistic  about  the  envisioned  capabilities  and  the  time  and  cost  it 
would  take  to  deliver.  As  much  as  concurrency  helped  this  endeavor,  more 
redesigns  and  rework  than  expected  occurred  during  initial  flight  training. 

Most  of  the  improvements  recommended  for  the  JSF  were  shown  in  Table 
2  to  correct  technical  deficiencies.  Unclosed  items  include  software  devel¬ 
opment,  the  HMDS,  weapons  accuracy  due  to  AESA  and  EOTS  issues, 
fueldraulics'  survivability,  and  fire  suppression.  All  of  these  were  pursued 
for  the  USAF's  IOC  declaration  in  August  2016,  five  years  later  than  planned 
(Insinna,  2016;  Pike,  2012).  On  the  positive  side,  for  any  great  leaps  in 
technology  for  the  United  States  to  maintain  tactical  advantages  in  aerial 
warfare,  we  just  have  to  try— because  even  temporary  failures  produce  gains 
in  knowledge. 

As  a  matter  of  relativeness  between  the  analysis  lanes,  Figure  6  shows  a 
summary  taxonomy  of  all  the  Harnessing  Technology  components'  con¬ 
tributing  factors  and  how  they  affect  the  pillars  of  acquisition  and  how 
the  pillars  of  acquisition  react  to  one  another  under  the  guise  of  “mature” 
technology.  Figure  6  is  visually  significant  in  showing  that  all  analysis  lanes 
(in  brown,  purple,  and  black)  contributed  several  factors  to  Technology 
Negative  (TN)  technology  that  had  numerous  negative  effects  on  schedule, 
cost,  and  performance.  As  for  recommendations,  the  green/positive  and  red/ 
negative  Technology  Recommendations  (TR)  point  to  a  split  Performance 
Recommendation  (PR).  Within  the  pillars  of  acquisition  olive  triangle, 
although  there  may  be  a  chance  that  new  technologies  can  significantly 
improve  performance  and  then  reduce  cost  and  decrease  schedule,  it  is  more 
likely  that  these  new  technologies  will  be  overpromised,  will  underperform, 
and  will  result  in  deficient  capabilities  due  to  immaturity  with  underdevel¬ 
oped  software  and  knowledge  gaps.  Government  program  managers  need 
to  do  abetter  job  in  assessing  technology  readiness  levels  of  new  systems. 
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FIGURE  6.  HARNESSING  TECHNOLOGY  SUMMARY  TAXONOMY 


Immature  Technology 
Immature  Manufacturing  Processes 
Not  Previously  Attempted 
Overpromised  &  Underperforming 
Capabilities 
Critical  Technologies’  Immaturity 
Complex  Software 
Weight  Control 
Competing  Flight  Test 
Manufacturing  Workarounds 
Overoptimistic,  Glossy,  Not  Tested 
Contractor’s  Risk  is  Contractually 
Limited 

Underdeveloped  Software 
Unachievable  Timelines 
Morphed  into  Multiblock  Program 
F-35B  STOVL  Weight 
HMDS  Weight  and  Display  Jitter 


Software  Development 
Realistic  Testing 
Fueldraulics  System 
Fire  Suppression 
Helmet  Mounted  Display  System 
Weapons  Accuracy 


Intangible  Gains^j 


Asymmetric  Advantage 
Sequential  Development 
Spiral  Blocks  of  Capabilities, 
Survive  Against  Advanced  Threat 
Numerous  Advanced  Technologies 
Most  Advanced,  Dominant  Aircraft 


Commercial-off-the-shelf 
Survive  Against  Advanced  Threat 
Numerous  Advanced  Technologies 
Potential  Dividends  in  Tactical  Advantages 


Services’  Contradictory  Aerodynamic  Requirements 
%  of  Common  Parts  Decreases  with  Time 
More  Stakeholders  Take  More  Time 
Threatens  Service  Autonomy 


Not  Enough  Testing  or  Appropriate  Testing 
Unplanned  Retrofits  as  Result  of  Testing 
Immature  Manufacturing 
Rework  Due  to  Simultaneous  Production 
Production  with  Immature  Systems 
Competing  Flight  Tests 


Weapons  System  to  Warfighter  Sooner 


Sequential  Development 
Spiral  Blocks  of  Capabilities 


Note.  Cl  =  Cost  Increased,  CIR  =  Cost  Increase  Reason,  CD  =  Cost  Decreased,  CR  =  Cost 
Recommendation,  SL  =  Schedule  Lengthened,  SLR  =  Schedule  Lengthened  Reason,  SS  = 
Schedule  Shortened,  SR  =  Schedule  Recommendation,  PRD  =  Performance  Reduced,  PRR 
=  Performance  Reduced  Reason,  PI  =  Performance  Improved,  Tl  =  Technology  Intent,  TN  = 
Technology  Negative,  TP  =  Technology  Positive,  TR  =  Technology  Recommendation,  CMN 
=  Commonality  Negative,  CCI  =  Concurrency  Intent,  CCN  =  Concurrency  Negative,  CCP  = 
Concurrency  Positive 


Red  =  Previous  Joint  Aircraft  Programs,  Blue  =  Government/Think  Tank  Reports  and 
Scholarly  Journals,  Black  =  Respondents’  Questionnaires 


SQ3  on  Demanding  Commonality 

The  original  intent  of  demanding  commonality  is  to  drive  down  devel¬ 
opment,  manufacturing,  and  sustainment  costs  by  having  a  high  level  of 
interchangeable  parts  and  software  for  a  weapon  system  to  reap  the  bene¬ 
fits  of  economies  of  scale  when  large  quantities  are  bought.  Commonality 
was  the  result  of  Balancing  Requirements  among  three  Services  into  a 
single,  multivariant  aircraft  that  has  about  40%  airframe  commonality,  but 
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almost  100%  commonality  in  mission  systems,  sustainment,  and  training 
software,  eventually  supporting  over  3,000  F-35s.  Although  the  airframe 
commonality  among  the  F-35A/B/C  is  not  as  high  as  the  80%  expected, 
there  is  still  just  one  manufacturing  line  that  can  produce  all  three  variants, 
significantly  lowering  the  price  per  unit,  according  to  annual  GAO  reports, 
which  became  attractive  to  foreign  nations.  Furthermore,  with  only  40% 
airframe  commonality,  most  of  the  parts  that  can  be  replaced  are  common 
and  that  will  save  hundreds  of  millions  of  dollars  over  the  decades  to  come. 
As  more  F-35s  are  built,  over  time  there  are  less  spare  parts  per  aircraft 
needed,  which  drives  logistical  costs  down.  Even  with  partner-unique  parts, 
the  commonality  of  the  computerized  maintenance  and  training  system 
based  on  the  ALIS  backbone  will  also  save  hundreds  of  millions  of  dollars 
once  ALIS  has  its  full  capability.  Furthermore,  beyond  the  airframe,  with 
almost  100%  common  mission  and  training  systems,  combining  tests  points 
shortens  the  schedule. 

Similar  to  the  Balancing  Requirements  previous  discussion  in  terms  of 
negativeness,  in  order  to  meet  requirements  within  the  different  Services' 
operating  environments,  compromise  entailed  making  sacrifices  in  KPPs. 
For  example,  the  USAF  accepted  a  less  than  optimal  ejection  system  with  a 
backwards  canopy  and  less  fuel  capacity  to  meet  USMC  F-35B  weight  needs. 
When  compromise  was  not  attained,  divergence  occurred.  For  example,  the 
USAF  F-35A  maintained  the  internal  gun,  while  the  USMC  F-35B  removed 
the  internal  gun  to  make  more  room  for  the  lift  fan  part  of  their  engine,  and 
the  USN  F-35C  has  a  third  larger  wing  than  the  F-35As  and  Bs.  These  sep¬ 
arate  designs  contradict  the  ideal  of  demanding  commonality.  A  common 
recommendation  from  the  managers  and  the  providers  is  to  keep  forcing 
performance-affordability  compromises  early  in  the  program,  but  even  the 
customers  admitted  that  Services'  autonomy  and  parochialism  usually  win 
at  a  cost  of  development  time. 

As  a  matter  of  relativeness  between  the  analysis  lanes,  Figure  7  shows  a 
summary  taxonomy  of  all  the  Demanding  Commonality  components'  con¬ 
tributing  factors,  and  how  they  affect  the  pillars  of  acquisition,  and  how  the 
pillars  of  acquisition  react  to  one  another  under  the  pressure  of  commonal¬ 
ity.  Figure  7  is  a  visual  “draw”  between  the  positives  and  negatives,  looking 
very  similar  to  those  of  Balancing  Requirements  in  Figure  5.  Most  of  the 
positive  factors  and  relationships,  however,  came  from  the  questionnaire 
responses  (in  black),  and  the  negatives  came  from  previous  joint  aircraft 
programs  (in  purple).  If  commonality  was  graded  over  time,  it  has  been 
losing  early  and  often,  evident  by  a  couple  of  Nunn-McCurdy  breaches,  but 
GAO  reports  and  several  respondents  believe  there  is  high  probability  that 
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over  time  the  story  will  change  to  be  a  successful  strategy.  Within  the  pillars 
of  acquisition  olive  triangle,  Demanding  Commonality  was  sold  as  a  cost¬ 
and  time-savings  strategy  to  make  three  Service  fighter  programs  into  one, 
while  the  same  mission  systems  and  simplified  parts  pool  will  eventually 
reap  a  cost  benefit.  However,  the  expectations  of  these  savings  were  oversold 
in  the  near  term,  and  the  Services  will  not  let  you  forget  that  they  sacrificed 
on  some  KPPs  to  attempt  to  meet  the  commonality  goals. 


FIGURE  7.  DEMANDING  COMMONALITY  SUMMARY  TAXONOMY 


One  Aircraft  to  Meet  Needs  of  Two  or  More  Services 
Economies  of  Scale 
Jointness  Trade-offs 
Sharing  Parts  Reduces  Life-cycle  Costs 
Affordable  &  Interchangeable 
Economies  of  Scale  of  Three  Programs  in  One 
Development/Sustainment  Efficiencies 
One-solution,  Interoperable,  Multivariant 
Smaller  Logistics,  Simplified  Supportability 


Combine  Programs  to  Get  One  Aircraft  ^ 

Jointness  Trade-offs 
One-program  Avoiding  Three 
Compatible  Variants 
Means  to  an  Ends  of  Commonality 

Invest  Into  One  Aircraft  to  Meet  " 
Multiservice  Needs 
Meet  Timeline  Milestones  for  All  Involved 
Combining  Test  Points 
Meet  Numerous  U.S.  Service  &  Foreign 
Countries’  Needs 
Combining  Test  Points 
Meet  Numerous  U.S.  Service  &  Foreign 
Countries’  Needs 

Joint  Programs  More  Expensive" 

Meeting  Each  Service’s  Needs 
F-35B’s  STOVL  Weight,  F-35A’s  Internal  Gun 
Services’  Unwillingness  to  Compromise 
Compromise  Delay  Allows  Threats  to  Counter 
Reduced,  Compromised,  or  Traded-off  KPPs^ 

Immature  Technology"!  i  i 
Immature  Manufacturing  Processes  r  TN 
Not  Previously  Attempted^ 

Different  Services’  Operating  Environments 

Services’ Changing  Requirements  j-|  BRN 
Driving  Consensus  Forces  Concessions 
Increased  Weight 

Force  Performance-affordability  \ 
Compromises  / 


Common  Parts  Simplifies 
Sustainment 
Economies  of  Scale 
More  Production  Orders,  Larger 
Parts  Pool 

Sharing  Parts  Reduces  Life-cycle 
Costs 

Streamlined  Logistics 
One  Program  vice  Three  Programs 
One  Production  Line  for  Three 
Variants 

Mission  Systems  More  Important 
Than  Structure 
Development,  Procurement, 
Sustainment  Efficiencies 


Services’  Contradictory  Aerodynamic  Requirements 

%  of  Common  Parts  Decreases  With  Time 

More  Stakeholders  Take  More  Time 

Threatens  Service  Autonomy 

Divergence  Amongst  Variants 

USN  F-35C  Environment  Challenges 

Untested  Integration 

Sacrifice  in  Capabilities 

Compromising  Takes  Time 

F-35B  Weight  Requirements 


Force  Performance-affordability  Compromises 


Note.  Cl  =  Cost  Increased,  CIR  =  Cost  Increase  Reason,  CD  =  Cost  Decreased,  CDR  = 

Cost  Decrease  Reason,  SL  =  Schedule  Lengthened,  SS  =  Schedule  Shortened,  PRD  = 
Performance  Reduced,  PRR  =  Performance  Reduced  Reason,  PI  =  Performance  Improved, 
BRI  =  Balancing  Requirements  Intent,  BRN  =  Balancing  Requirements  Negative,  BRP  = 
Balancing  Requirements  Positive,  BRR  =  Balancing  Requirements  Recommendation,  CMI  = 
Commonality  Intent,  CMN  =  Commonality  Negative,  CMP  =  Commonality  Positive,  CMR  = 
Commonality  Recommendation 


Red  =  Previous  Joint  Aircraft  Programs,  Blue  =  Government/Think  Tank  Reports  and 
Scholarly  Journals,  Black  =  Respondents’  Questionnaires 
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SQ4  on  Evoking  Concurrency 

The  original  intent  of  evoking  concurrency  was  to  systematically  over¬ 
lap  development,  production,  test,  and  fielding  of  a  weapon  system  to  get 
it  quickly  to  the  warfighters  with  some  capability  in  order  to  improve  fol- 
low-on  production  lots  based  on  what  was  learned  from  the  warfighters,  thus 
saving  time  and  money.  Concurrency  was  offered  as  a  strategy  to  answer  the 
1980s  Packard  Commission  about  how  to  prevent  long  and  costly  defense 
acquisition  programs  (Eide  &  Allen,  2012).  In  theory,  concurrency  short¬ 
ens  the  program  by  having  test  and  training  conducted  simultaneously.  It 
should  also  force  the  program  to  account  for  supportability  and  logistics 
earlier.  Several  respondents  explained  how  concurrency  was  planned  for  4 
to  6  years  over  three  blocks  of  aircraft— Blocks  1,  2,  and  3,  but  the  program 
ended  up  with  11  different  blocks— Blocks  0, 1A,  IB,  2A,  2B,  3i,  3F,  4.1,  4.2, 

4.3,  and  4.4.  By  2011,  the  JPO's  plan  for  U.S.  F-35s  was  to  retrofit  25%  of  the 
fleet  based  on  procuring  600  aircraft  before  the  end  of  initial  operational 
test  and  evaluation  in  March  2012  (Blickstein  et  al.,  2011).  In  comparison, 
the  F-lll  produced  141  F-llls  for  the  USAF  before  changes  could  be  made  on 
the  manufacturing  line  for  a  25%  concurrency  rate  against  547  total  USAF 
F-llls  (Coulam,  1977;  Richey,  2005).  The  final  concurrency  rate  will  not  be 
known  until  after  the  fact. 

The  1980s  JVX  V-22  relied  heavily  on  concurrency.  It  entered  full-rate 
production  without  ensuring  mature  manufacturing  processes  that  required 
a  redesign  and  retrofit  of  hydraulics  and  electrics  for  the  tilt-rotor  system, 
resulting  in  a  Nunn-McCurdy  breach  in  2001,  after  the  USMC  had  begun 
M  V-22  aircrew  training  in  2000  (General  Accounting  Office,  2003;  Whittle, 
2010).  Seven  years  later,  the  USMC  declared  MV-22  IOC  in  2007  (Whittle, 
2010).  There  is  little  argument  that  with  concurrency  more  F-35  issues  have 
been  found  earlier  than  otherwise  predicted,  and  many  issues  have  been 
corrected  quicker  with  cut-ins  on  the  production  line,  but  the  amount  and 
complexity  of  the  retrofits  exceeded  expectations.  Most  of  this  disappoint¬ 
ment  is  due  to  incomplete,  late  flight  testing  and  overpromised  capabilities. 
Although  the  USAF  started  training  in  2012  with  an  aircraft  of  very  limited 
capability,  right  after  the  second  and  last  Nunn-McCurdy  breach,  IOC  will 
not  be  declared  until  2016.  Instead  of  dealing  with  three  blocks  of  aircraft 
during  these  4  years,  one  “operator”  respondent  emphasized  that  pilots  and 
maintainers  will  have  dealt  with  seven  blocks— Block  0.5, 1A,  IB,  2A,  2B,  3i 
(small  “i”  for  initial  combat  capability),  and  3F.  Follow-on  F-35  development 
will  include  four  more  blocks  starting  in  2019  with  one  every  2  years— Block 

4.1. 4.2. 4.3,  and  4.4  (Tirpak,  2015). 
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One  provider  said  this  about  JSF  concurrency :  “The 
original  model  for  JSF  is  solid .  Build  test  aircraft, 
build  training  aircraft  while  testing  is  going  on, 
and  then  accelerate  to  full  rate  production  when  the 
aircraft  has  matured  sufficiently.” 

Unfortunately,  Concurrency  Recommendations  (CCR)  did  not  have  any 
strong  associations  for  the  Reports'/Journals'  co-occurrence/c-coefficient 
tables  in  Figure  2 ,  but  the  left  table  for  the  Respondents'  co-occurrence/c-co- 
efficient  tables  in  Figure  4  showed  they  required  further  investigation. 
It  suggested  that  program  managers  should  be  more  disciplined  when 
accepting  immature  technology  and  be  more  suspicious  of  the  promises 
of  capabilities  early  on.  By  digging  deeper  into  the  respondents'  question¬ 
naires,  some  appropriate  recommendations  were  found.  A  few  manager 
respondents  suggested  ensuring  the  technology  readiness  levels  are  more 
accurate,  reserving  concurrency  for  a  less  complex  weapon  system,  and  to 
never  accept  this  level  of  concurrency  again.  Several  customer  respondents 
suggested  making  the  early  production  lots  smaller  in  quantity.  One  provider 
said  this  about  JSF  concurrency:  “The  original  model  for  JSF  is  solid.  Build 
test  aircraft,  build  training  aircraft  while  testing  is  going  on,  and  then  accel¬ 
erate  to  full  rate  production  when  the  aircraft  has  matured  sufficiently.'' 

As  a  matter  of  relativeness  between  the  analysis  lanes,  Figure  8  shows 
a  summary  taxonomy  of  all  the  Evoking  Concurrency  components'  con¬ 
tributing  factors,  and  how  they  affect  the  pillars  of  acquisition,  and  how 
the  pillars  of  acquisition  react  to  one  another  under  the  pressure  of  com¬ 
monality.  Factors  contributing  to  Concurrency  Negative  (CCN)  are  more 
apparent  than  to  Concurrency  Positive  (CCP).  All  analysis  lanes  (previous 
in  brown,  reports/journals  in  purple,  and  questionnaires  in  black)  signifi¬ 
cantly  contributed  to  the  concurrency  negative  impacts  on  schedule,  cost, 
and  performance.  Within  the  pillars  of  acquisition  olive  triangle,  there  are 
dueling  positive  and  negative  component  triangles  between  schedule,  cost, 
and  performance.  The  positive  Schedule  Shortened  (SS)-Cost  Decreased 
(CD) -Performance  Improved  (PI)  triangle  assumes  mature  technology  and 
expects  to  wring  out  problems  sooner  to  get  a  better  aircraft  sooner.  The  neg¬ 
ative  Schedule  Lengthened  (SL)-Cost  Increased  (Cl) -Performance  Reduced 
(PRD)  triangle  harps  on  overpromised  results  and  underperforming  reality 
that  was  reiterated  by  the  reports/journal  (in  purple)  and  the  respondents' 
questionnaire  analyses  (in  black). 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


149 


Scandal  and  Tragedy? 


http://www.dau.mil 


FIGURE  8.  EVOKING  CONCURRENCY  SUMMARY  TAXONOMY 
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Sequential  Development 
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Feedback  to  Production 
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Do  Not  Overpromise 
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Immature  Manufacturing 
Rework  Due  to  Simultaneous  Production 
Production  with  Immature  systems 
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Late  Development  &  Test  Exacerbates  Strategy 


Note.  AOL  =  Aircraft  Operational  Limits,  Cl  =  Cost  Increased,  CIR  =  Cost  Increase 
Reason,  CD  =  Cost  Decreased,  CR  =  Cost  Recommendation,  SL  =  Schedule  Lengthened, 
SS  =  Schedule  Shortened,  SSR  =  Schedule  Shortened  Reason,  SR  =  Schedule 


Recommendation,  PRD  =  Performance  Reduced,  PI  =  Performance  Improved,  Tl 
=  Technology  Intent,  TN  =  Technology  Negative,  TP  =  Technology  Positive,  CCI  = 
Concurrency  Intent,  CCN  =  Concurrency  Negative,  CCP  =  Concurrency  Positive,  CCR  = 
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Red  =  Previous  Joint  Aircraft  Programs,  Blue  =  Government/Think  Tank  Reports  and 
Scholarly  Journals,  Black  =  Respondents’  Questionnaires 


SQ5  on  Encouraging  International  Partnering 

The  original  intent  of  encouraging  partnering  was  to  get  partner  nations 
invested  early  in  the  program  to  help  pay  for  development  and  to  order  and 
receive  aircraft  early  in  the  program.  This  would  reduce  the  price  per  unit 
and  result  in  coalition  interoperability.  In  return  for  taking  risk  by  investing 
in  development,  the  Partners  get  to  learn  how  to  develop  a  5th  Gen  aircraft 
that  no  other  opportunity  would  provide.  Over  the  life  of  the  40-year  pro¬ 
gram,  sustainment  costs  are  also  shared.  As  a  business  model  to  defend 
against  U.S.  congressional  budget  cutters,  international  support  goes  a  long 
way,  especially  when  the  expectation  is  to  train  and  fight  together. 
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Most  respondents  referred  to  building  partner  capacity  as  the  positive 
by-product  of  partnering— it  promotes  the  United  States'  interest  in  not 
fighting  alone,  in  fighting  on  the  same  level  on  the  same  side,  and  in  spread¬ 
ing  the  burden  of  policing  the  world.  Besides  reducing  U.S.  development 
and  production  costs  and  building  partnership  capacity  in  the  expectation 
we  will  fight  together,  partner  nations  reap  the  added  benefits  of  economic 
opportunity  in  their  own  countries  as  their  companies  become  suppliers 
for  LM.  As  one  of  the  first  partners,  U.K.'s  Martin-Baker  will  manufacture 
all  the  F-35A/B/Cs'  ejection  system,  representing  billions  of  dollars  in  rev¬ 
enue  to  outfit  over  3,000  potential  aircraft  (JPO,  2014).  Italy  gained  10,000 
jobs  created  by  their  industrial  participation  in  the  program  including  the 
activities  of  the  Cameri  Final  Assembly  Check-Out,  Maintenance,  Repair, 
Overhaul,  and  Upgrade  (MRO&U)  plant  to  assemble  their  90  F-35s  via 
license  from  LM  (Marrone,  2013).  Even  Turkey  was  approved  to  be  the 
first  of  three  nations  to  oversee  MRO&U  for  the  F135  engine  (Butler,  2015). 
Partner  countries  also  get  to  bid  on  managing  spare  parts  logistics  hubs  for 
Europe  and  Asia  (Butler,  2015). 

This  arrangement,  guided  by  the  JPO  (2007)  Production ,  Sustainment ,  and 
Follow-on  Development  (PSFD)  Memorandum  of  Understanding  (MOU) 
signed  by  the  partner  nations  listed  in  the  MOU's  Table  3  (Chapter  2), 
is  a  first-of-a-kind  partnership  that  comes  with  several  challenges  not 
experienced  in  normal  FMS.  In  FMS  cases,  the  weapon  system  is  already 
developed,  there  is  extra  capacity  to  produce  and  to  train  if  requested,  and 
the  United  States  usually  dictates  the  terms.  In  this  partnership,  no  matter 
how  many  aircraft  a  nation  is  purchasing,  the  foreign  countries  get  an  equal 
vote  right  from  the  start,  and  when  priorities  cannot  be  agreed  upon,  it  takes 
longer.  If  a  country  has  a  unique  requirement,  although  it  pays  for  it  alone,  it 
takes  nonrecurring  engineering  time  away  from  other  design  and  test  pri¬ 
orities.  But  the  most  difficult  challenge  concerning  the  strong  relationship 
between  cost  and  schedule  is  the  problem  of  getting  classified  capabilities 
released  to  partner  nations.  This  is  especially  important  when  USAF  and 
foreign  F-35A  student  pilots  will  be  training  with  one  another,  taught  by  one 
another  by  USAF  and  Partner  instructors,  and  flying  each  other's  aircraft 
at  Luke  AFB— a  process  known  as  “pooling.”  Although  there  are  DoD  offices 
in  charge  of  protecting  critical  technologies,  five  customer  respondents 
commented  that  other  disclosure,  security,  and  exportability  offices  have 
not  fully  embraced  or  understood  the  PSFD  MOU's  intent.  Improvements 
to  remedy  these  negatives  come  strictly  from  respondents'  questionnaires 
and  start  with  recommending  delaying  partnering  until  development  is 
complete,  settling  disclosure  issues  before  partners  invest,  and  making 
voting  proportional  to  the  number  of  aircraft  ordered. 
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As  a  matter  of  relativeness  between  the  analysis  lanes,  Figure  9  shows  a 
summary  taxonomy  of  all  the  Encouraging  Partnering  components'  con¬ 
tributing  factors  and  how  they  affect  the  pillars  of  acquisition  and  how 
the  pillars  of  acquisition  react  to  one  another  under  the  pressure  of  com¬ 
monality.  Although  Figure  9  looks  visually  even,  the  contributing  factors 
to  Partnering  Positive  (PTP)  of  building  partner  capacity,  interoperabil¬ 
ity,  and  true  coalition  warfare,  and  the  tangible,  international  economic 
opportunities  brought  forward  mostly  by  the  reports/journals  (in  purple) 
and  the  respondents'  questionnaires  (in  black),  outweigh  the  procedural 
frustrations  in  Partnering  Negative  (PTN)  of  releasability  and  voting  that 
will  be  solved  over  time  by  necessity.  However,  there  will  be  a  price  to  pay 
in  time  and  cost  within  the  pillars  of  acquisition  olive  triangle.  This  is  espe¬ 
cially  true  with  unique  country  requests  and  the  fact  that  partner  nations 
all  have  civilian  governments  that  may  delay  ordering  and  purchasing  from 
the  previously  agreed  plan,  which  instantly  raises  the  price  per  unit  that 
particular  year. 


FIGURE  9.  ENCOURAGING  PARTNERING  SUMMARY  TAXONOMY 


Partner  After  Some  Developing  &  Testing 
Have  Clear  Understanding  of  Partnering 
Disclosure  Approved  Beforehand 
Proportional  Voting 


More  Countries  Have  More  Inputs  Sooner 
Classified  Capabilities  Release  Process 
Unique  Country  Requirements 
Equal  Voices  in  Determining  Prioritizations 
Partner  Budgeting  &  Delayed  Partner  Buys 
Unique,  First-of-a-kind  Partnership 


More  Countries  Buy  More  Sooner 
Interoperability 
Industry  Shared 
Reduced  Logistical  Footprint 
International  Economic  Opportunities 
Multiple  Countries  Investing 
Reducing  Price  Per  Unit 
Building  Partner  Capacity 


Desire  Coalition  Interoperability 
More  Countries,  More  Orders 
Global  Supply  System 
Sharing  &  Spreading  Development  Costs 
Reduce  Early  Production  Price  Per  Unit 
Share  Long-term  Sustainment  Costs 
Synergy  of  Training  Together 


Note.  Cl  =  Cost  Increased,  CIR  =  Cost  Increase  Reason,  CD  =  Cost  Decreased,  CDR  = 

Cost  Decrease  Reason,  SL  =  Schedule  Lengthened,  SLR  =  Schedule  Lengthened  Reason, 
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PI  =  Performance  Improved,  PTI  =  Partnering  Intent,  PTP  =  Partnering  Positive,  PTN  = 
Partnering  Negative,  PTR  =  Partnering  Recommendation 

Red  =  Previous  Joint  Aircraft  Programs,  Blue  =  Government/Think  Tank  Reports  and 
Scholarly  Journals,  Black  =  Respondents’  Questionnaires 


Final  Recommendations 

All  program  managers  in  the  defense  industry  and  in  commercial  busi¬ 
ness  should  note  the  relations  within  the  pillars  of  acquisition  that  were 
confirmed.  Although  the  JSF  program  is  in  the  21st  century,  lessons  on  the 
pillars  of  acquisition  were  first  relearned  in  the  20th  century  with  the  TFX, 
JVS,  and  JPATS  programs  and  reconfirmed  by  analysis  on  the  JSF  program. 
If  technical  performance  cannot  be  sacrificed,  but  is  hard  to  achieve,  expect 
cost  to  rise  and  schedule  to  lengthen.  If  cost  is  fixed  due  to  budgeting  and  the 
schedule  is  expected  to  stay  the  same,  performance  will  suffer  and  require¬ 
ments  will  be  lowered.  If  a  program  manager  wants  to  maintain  schedule, 
cost  usually  goes  up  for  more  personnel,  but  performance  may  suffer  again 
due  to  rushing. 

What  is  learned  is  not  necessarily  about  scandal  and  tragedy;  however,  all 
program  managers  in  the  defense  industry  will  recognize  familiar  lessons 
and  learn  some  new  ones: 

•  Balancing  requirements  between  the  Services  is  not  recom¬ 
mended  for  large  joint  MDAPs,  especially  for  aircraft.  However, 
balancing  requirements  within  a  single-Service  program  to 
meet  cost,  schedule,  and  performance  goals  should  be  expected. 

This  takes  disciplined  leadership  and  the  ability  to  manage 
expectations  with  transparency  of  what  can  be  achieved. 

•  Harnessing  technology,  or  reaching  for  immature  technology, 
should  be  done  only  if  needed  to  maintain  a  tactical  advantage 
in  a  strategic  environment.  Otherwise,  program  managers 
need  to  assess  technology  readiness  levels  better  and  not  be  so 
enamored  with  contractors'  glossy  brochures  of  cost,  schedul¬ 
ing,  and  performance. 

•  Demanding  commonality  may  be  costly  and  time  consum¬ 
ing  as  the  result  of  balancing  requirements  in  the  near  term, 
but  should  pay  off  with  a  streamlined  logistics  system  in  the 
long  term.  Do  not  underestimate  the  commonality  of  mission 
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systems  (communications,  sensors,  mission  planning,  and 
prognostic  health  management)  and  of  training  systems 
(courseware  and  simulators). 

•  Evoking  concurrency  as  a  primary  strategy  will  be  disap¬ 
pointing  if  technology  was  incorrectly  represented  as  mature, 
but  this  is  a  matter  for  better  discipline,  management,  and 
execution,  and  it  cannot  be  addressed  with  Flyvbjerg's  (2003) 
megaprojects*  paradox  that  the  program  is  too  big  to  fail.  If  con¬ 
currency  occurs  because  a  development  program  falls  behind 
schedule,  project  managers  need  to  adjust  expectations,  but 
that  still  requires  discipline. 

•  Encouraging  partnering  needs  to  be  understood  better  from  the 
beginning  and  embraced  by  government  agencies  and  military 
services  in  terms  of  disclosure,  security,  and  exportability— 
especially  if  the  United  States  truly  wants  to  build  partner 
capacity  and  to  have  true,  interoperable,  coalition  warfare. 


154 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


January  2017 


References 

Air  Education  and  Training  Command.  (2010,  March).  A  training  system  for  the  21st 
century:  JPATS  and  the  T-6A  (AFD-061213-005).  Retrieved  from  https://www. 
rairarubiabooks.com/related-pdf-enter-trainer-extended-wings.html 

Antill,  P.,  &  Ito,  P.  (2013).  The  UK  and  the  Joint  Strike  Fighter:  The  trials  and  tribulations 
of  international  collaborative  procurement.  International  Journal:  Canada’s 
Journal  of  Global  Policy  Analysis,  68(1),  13-29.  doi:10.1177/002070201306800103 

Art,  R.  (1969).  The  TFX  decision:  McNamara  and  the  military.  Boston,  MA:  Little,  Brown 
and  Company. 

Associated  Press.  (2016,  April  26).  Senator  says  fighter  program  has  been  scandal  and 
tragedy.  Retrieved  from  http://www.foxnews.com/us/2016/04/26/senator-says- 
fighter-program-has-been-scandal-and-tragedy.html 

Blickstein,  I.,  Boito,  M.,  Drezner,  J.,  Dryden,  J.,  Florn,  K.,  Kallimani,  J.,  ...  Wong,  C.  (2011). 
Root  cause  analyses  of  Nunn-McCurdy  breaches:  Zumwalt  Class  Destroyer,  Joint 
Strike  Fighter,  Longbow  Apache,  and  Wideband  Global  Satellite  ( Vol .  1).  Retrieved 
from  RAND  Corporation  website:  http://www.rand.org/content/dam/rand/pubs/ 
monographs/2011/R  AN  D_MG117T1.pdf 

Blickstein,  I.,  Nemfakos,  C.,  &  Sollinger  J.  M.  (2013).  Digging  out  the  root  cause: 
Nunn-McCurdy  breaches  in  major  defense  acquisition  programs.  Defense 
Acquisition  Research  Journal,  20(2),  128-153.  Retrieved  from  http://www.dau.mil/ 
publications/DefenseARJ/ARJ/ARJ66/ARJ_66-BI  ickstein.pdf 

Butler,  A.  (2015,  January  6).  F-35  nearing  in-service  debuts:  With  a  handful  of  fighters 
nearing  operations,  F-35  is  still  the  one  to  beat.  Aviation  Week  and  Space 
Technology,  176(47 ),  69.  Retrieved  from  http://aviationweek.com/defense/j-10b- 
f-35-nearing-service-debuts 

Coulam,  R.  (1977).  Illusions  of  choice:  The  F-lll  and  the  problem  of  weapons 
acquisition  reform.  Princeton,  NJ:  Princeton  University  Press. 

Creswell,  J.  (2009).  Research  design:  Qualitative,  quantitative,  and  mixed  methods 
approaches  (3rd  ed.).  Thousand  Oaks,  CA:  Sage. 

Defense  Acquisition  University.  (2006).  Risk  management  guide  for  DoD  acquisition 
(6th  ed.).  Retrieved  from  http://www.dau.mil/publications/publicationsDocs/ 
RMG%206Ed%20Aug06.pdf 

Department  of  Defense.  (2012).  Joint  Primary  Aircraft  Training  System  (JPATS) 
Selected  Acquisition  Report  (SAR)  (RCS:  DD-A&T(Q&A)823-560).  Retrieved 
from  http://www.globalsecurity.org/military/library/budget/fy2012/sar/jpats_ 
december_2012_sar.pdf 

Department  of  Defense  Inspector  General.  (2013,  September  30).  Quality  assurance 
assessment  of  the  F-35  Lightning  II  program.  Retrieved  from  http://www.dodig. 
mil/pubs/documents/DODIG-2013-140.pdf 

Director,  Operational  Test  and  Evaluation.  (2004).  Fiscal  year  2004  annual  report 
F-35  Joint  Strike  Fighter  (JSF).  Retrieved  from  http://www.dote.osd.mil/pub/ 
reports/FY2004/pdf/af/2004F-35JSF.pdf 

Director,  Operational  Test  and  Evaluation.  (2005).  Fiscal  year  2005  annual  report 
F-35  Joint  Strike  Fighter  (JSF).  Retrieved  from  http://www.dote.osd.mil/pub/ 
reports/FY2005/pdf/af/2005f-35.pdf 

Director,  Operational  Test  and  Evaluation.  (2006).  Fiscal  year  2006  annual  report 
F-35  Joint  Strike  Fighter  (JSF).  Retrieved  from  http://www.dote.osd.mil/pub/ 
reports/FY2006/pdf/af/2006f35.pdf 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


155 


Scandal  and  Tragedy? 


http://www.dau.mil 


Director,  Operational  Test  and  Evaluation.  (2007a).  Fiscal  year  2007  annual  report 
F-35  Joint  Strike  Fighter  (JSF).  Retrieved  from  http://www.dote.osd.mil/pub/ 
reports/FY2007/pdf/af/2007f35.pdf 

Director,  Operational  Test  and  Evaluation.  (2007b).  Joint  Primary  Aircraft  Training 
System  ( JPATS ).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2007/ 
pdf/navy/2007jpats.pdf 

Director,  Operational  Test  and  Evaluation.  (2008).  F-35  Lightning  it  Joint  Strike 
Fighter  (JSF).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2008/ 
pdf/dod/2008f35.pdf 

Director,  Operational  Test  and  Evaluation.  (2009).  F-35  Lightning  ii  Joint  Strike 
Fighter  (JSF).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2009/ 
pdf/dod/2009f35.pdf 

Director,  Operational  Test  and  Evaluation.  (2010).  F-35  Joint  Strike  Fighter 

(JSF).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2010/pdf/ 
dod/201  Of35jsf.pdf 

Director,  Operational  Test  and  Evaluation.  (2011).  F-35  Joint  Strike  Fighter 

(JSF).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2011/pdf/ 
dod/2011f35jsf.pdf 

Director,  Operational  Test  and  Evaluation.  (2012).  F-35  Joint  Strike  Fighter 

(JSF).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2012/pdf/ 
dod/2012f35jsf.pdf 

Director,  Operational  Test  and  Evaluation.  (2013).  F-35  Joint  Strike  Fighter 

(JSF).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2013/pdf/ 
dod/2013f35jsf.pdf 

Director,  Operational  Test  and  Evaluation.  (2014).  F-35  Joint  Strike  Fighter 

(JSF).  Retrieved  from  http://www.dote.osd.mil/pub/reports/FY2014/pdf/ 
dod/2014f35jsf.pdf 

Dunne,  T.  (2011).  A  case  for  the  F-35  Lightning.  National  Defence  and  the  Canadian 
Armed  Forces,  77(4),  55-60.  Retrieved  from  http://www.journal.forces.gc.ca/vo11/ 
no4/55-dunne-eng.asp 

Eide,  P.  K.,  &  Allen,  C.  D.  (2012).  The  more  things  change,  acquisition  reform  remains 
the  same.  Defense  Acquisition  Research  Journal,  79(1),  99-120.  Retrieved  from 
http://www.dau.mil/pubscats/PubsCats/AR%20Journal/arj61/Eide61.pdf 

Ergas,  H.  (2009).  Trillions  for  military  technology:  How  the  Pentagon  innovates  and 
why  it  costs  so  much.  Agenda:  A  Journal  of  Policy  Analysis  and  Reform,  16(2),  95- 
98.  Retrieved  from  http://search. proquest. com/docview/209796602?accountid= 
28180 

Everstine,  B.  (2015,  April  25).  Problems  plaguing  F-35’s  next-gen  maintenance  system. 
Retrieved  from  http://www.airforcetimes.com/story/military/2015/04/15/ 
problems-facing-f35-maintainers-automated-system/25781075/ 

Flyvbjerg,  B.  (2003).  Megaprojects  and  risk:  An  anatomy  of  ambition.  Cambridge,  UK: 
Cambridge  University  Press. 

Friese,  S.  (2013).  ATLAS,  ti  7  user  guide  and  reference.  Berlin,  Germany:  ATLAS. ti 

Scientific  Software  Development.  Retrieved  from  http://atlasti.com/wp-content/ 

uploads/2014/05/atlasti_v7_manual_201312.pdf?q=/uploads/media/atlasti_v7_ 

manual_201312.pdf 

Gantt,  M.  E.  (2002).  Strapping  in  and  baiting  out:  Navy  and  Air  Force  joint  acquisition 
of  aircraft.  Retrieved  from  Defense  Technical  Information  Center  website:  http:// 
www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA420511 


156 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


January  2017 


Gates,  R.  (2009,  December  2).  Statement  on  Afghanistan  to  the  Senate  Armed 
Services  Committee.  Retrieved  from  http://archive.defense.gov/speeches/ 
default.  aspx?mo=12&yr=2009 

General  Accounting  Office.  (1973).  F-111  Aircraft.  Retrieved  from  http://www.gao.gov/ 
assets/80/78812,  pdf 

General  Accounting  Office.  (1986).  DoD  acquisition:  Case  study  of  the  Navy  V-22 

Osprey  joint  vertical  lift  aircraft  program  (GAO-NSIAD-86-45S-7).  Retrieved  from 
http://www.gao.gov/assets/210/208624.pdf 

General  Accounting  Office.  (1990).  Navy  aviation:  The  V-22— Progress  and  problems 
(GAO/NS  I  AD -91 -45).  Retrieved  from  http://www.gao.gov/assets/220/213295.pdf 

General  Accounting  Office.  (1994).  V-22  development— Schedule  extended, 

performance  reduced,  and  costs  increased  (GAO/NSIAD-94-14).  Retrieved  from 
http://www.gao.gov/assets/220/218986.pdf 

General  Accounting  Office.  (1997).  Navy  aviation:  V-22  cost  and  capability  to  meet 
requirements  are  yet  to  be  determined  (GAO/NSI  AD-98-13).  Retrieved  from 
http://www.gao.gov/assets/230/224788.pdf 

General  Accounting  Office.  (2003).  Defense  acquisitions:  Assessments  of  major 
weapon  programs  (GAO-03-476).  Retrieved  from  http://www.gao.gov/ 
assets/240/238188. pdf 

General  Accounting  Office.  (2004).  Defense  acquisitions:  Assessments  of  major 
weapon  programs  (GAO-04-248).  Retrieved  from  http://www.gao.gov/ 
assets/250/241798. pdf 

Gertler,  J.  (2011).  V-22  Osprey  tilt-rotor  aircraft:  Background  and  issues  for  Congress 
(RL31384).  Retrieved  from  http://www.fas.org/sgp/crs/weapons/RL31384.pdf 

Government  Accountability  Office.  (2005).  Defense  acquisitions:  Assessments  of 

selected  major  weapon  programs  (GAO-05-301).  Retrieved  from  http://www.gao. 
gov/assets/250/245884. pdf 

Government  Accountability  Office.  (2006).  Defense  acquisitions:  Assessments  of 

selected  major  weapon  programs  (GAO-06-391).  Retrieved  from  http://www.gao. 
gov/assets/250/249468,  pdf 

Government  Accountability  Office.  (2007).  Defense  acquisitions:  Assessments  of 
selected  weapon  programs  (GAO-07-406SP).  Retrieved  from  http://www.gao. 
gov/assets/260/258416,  pdf 

Government  Accountability  Office.  (2008).  Defense  acquisitions:  Assessments  of 
selected  weapon  programs  (GAO-08-467SP).  Retrieved  from  http://www.gao. 
gov/assets/280/274156,  pdf 

Government  Accountability  Office.  (2009).  Defense  acquisitions:  Assessments 
needed  to  address  V-22  aircraft  operational  and  cost  concerns  to  define 
future  investments  (GAO-09-482).  Retrieved  from  http://www.gao.gov/ 
assets/290/289395. pdf 

Government  Accountability  Office.  (2010).  Defense  acquisitions:  Assessments  of 
selected  weapon  programs  (GAO-10-388SP).  Retrieved  from  http://www.gao. 
gov/assets/310/302379,  pdf 

Government  Accountability  Office.  (2011a).  Defense  acquisitions:  Assessments  of 

selected  weapon  programs  (GAO-11-233SP).  Retrieved  from  http://www.gao.gov/ 
assets/320/317081. pdf 

Government  Accountability  Office.  (2011b).  Joint  Strike  Fighter:  Restructuring  places 
program  on  firmer  footing,  but  progress  is  still  lagging  (GAO-11-677T).  Retrieved 
from  http://www.gao.gov/assets/130/126245.pdf 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


157 


Scandal  and  Tragedy? 


http://www.dau.mil 


Government  Accountability  Office.  (2012a).  Defense  acquisitions:  Assessments  of 
selected  weapon  programs  (GAO-12-400SP).  Retrieved  from  http://www.gao. 
gov/assets/590/589695,  pdf 

Government  Accountability  Office.  (2012b).  Joint  Strike  Fighter:  Restructuring  added 
resources  and  reduced  risk,  but  concurrency  is  still  a  major  concern  (GAO -12- 
525T).  Retrieved  from  http://www.gao.gov/assets/590/589454.pdf 

Government  Accountability  Office.  (2012c).  Joint  Strike  Fighter:  Restructuring  added 
resources  and  reduced  risk,  but  concurrency  is  still  a  major  concern  (GAO -12- 
525T).  Retrieved  from  http://www.gao.gov/assets/590/589454.pdf 

Government  Accountability  Office.  (2013).  Defense  acquisitions:  Assessments  of 
selected  weapon  programs  (GAO-13-294SP).  Retrieved  from  http://www.gao. 
gov/assets/660/653379,  pdf 

Government  Accountability  Office.  (2014).  Defense  acquisitions:  Assessments  of 
selected  weapon  programs  (GAO-14-340SP).  Retrieved  from  http://www.gao. 
gov/assets/670/662184,  pdf 

Government  Accountability  Office.  (2015).  Defense  acquisitions:  Assessments  of 
selected  weapon  programs  (GAO-15-342SP).  Retrieved  from  http://www.gao. 
gov/assets/670/668986,  pdf 

Hunter,  A.  (1998).  TSR2  with  hindsight.  Brighton,  U.K.:  RAF  Historical  Society. 

Insinna,  V.  (2016,  August  2).  Air  Force  declares  F-35A  ready  for  combat.  Defense 
News.  Retrieved  from  http://www.defensenews.com/story/breaking- 
news/2016/08/02/f35-ioc-air-force-operational-acc-combat/87948142/ 

Jogulu,  U.,  &  Pansiri,  J.  (2011).  Mixed  methods:  A  research  design  for  management 

doctoral  dissertations.  Management  Research  Review  34(6),  687-701.  doi:10.1108/ 
01409171111136211 

Joint  Program  Office.  (2007).  JSF production,  sustainment,  and  follow-on 

development  memorandum  of  understanding.  Retrieved  from  http://www.jsf.mil/ 
downloads/documents/JSF_PS  FD_MOU_-_07_Feb_07.pdf 

Logan,  D.  (1998).  General  Dynamics  F-111  Aardvark.  Atglen,  PA:  Schiffer. 

Leedy,  P.,  &  Ormrod,  J.  (2009).  Practical  research:  Planning  and  design  (9th  ed.). 

Upper  Saddle  River,  NJ:  Pearson. 

Lorell,  M.  A.,  Kennedy,  L.,  Leonard,  R.  S.,  Munson,  K.,  Abramzon,  S.,  An,  D.  L.,  & 

Guffey,  R.  A.  (2013).  Do  joint  fighter  programs  save  money?  Retrieved  from 
Rand  Corporation  website:  http://www.rand.org/content/dam/rand/pubs/ 
monographs/MG1200/MG1225/R  AN  D_MG1225.pdf 

Marrone,  A.  (2013).  Italy  and  the  F-35:  Rationales  and  costs.  International 

Journal:  Canada’s  Journal  of  Global  Policy  Analysis,  68(1),  31-48.  doi:10.1177/ 
002070201306800104 

McCullough,  A.  (2015,  July  20).  First  ospreys  for  Japan.  Air  Force  Magazine:  Online 
Journal  of  the  Air  Force  Association.  Retrieved  from  http://www.airforcemag. 
com/D  R  Archive/Pages/201 5/Ju  ly%202015/Ju  ly%2015%20201 5/Fi  rst-Ospreys- 
for-Japan.aspx 

Melese,  F.,  Franck,  R.,  Angelis,  D.,  &  Dillard,  J.  (2007).  Applying  insights  from 
transaction  cost  economics  to  improve  cost  estimates  for  public  sector 
purchases:  The  case  of  U.S.  military  acquisition.  International  Public  Management 
Journal,  70(4),  357-385.  doi:10.10  80/10967490701683511 

Miller,  J.  (1982).  General  Dynamics  F-W  “Aardvark.”  Fallbrook,  CA:  Aero  Publishers. 

Moustakas,  C.  (1990).  Heuristic  research:  Design,  methodology,  and  applications. 
Newbury  Park,  CA:  Sage. 

Patton,  M.  (2002).  Qualitative  research  and  evaluation  methods.  Thousand  Oaks,  CA: 
Sage. 


158 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


January  2017 


Pike,  J.  (2011).  F-35  Joint  Strike  Fighter  (JSF)  commonality.  Retrieved  from  the 
GlobalSecurity.Org  website:  http://www.globalsecurity.org/military/systems/ 
aircraft/f-35-commonality.htm 

Pike,  J.  (2012).  F-35  Joint  Strike  Fighter  (JSF)  Lightning  it  program.  Retrieved  from  the 
GlobalSecurity.  Org  website:  http://www.globalsecurity.org/military/systems/ 
aircraft/f-35-program.htm 

President’s  Blue  Ribbon  Commission  on  Defense  Management.  (1986).  A  quest 
for  excellence— final  report  to  the  President.  Retrieved  from  http://usacac. 
army.mil/cac2/CSI/docs/Gorman/06_Retired/01_Retired_1985_90/07_86_ 
PackardCommission_FinalReport/01_PackardCommission_FinalReport.pdf 

Richey,  G.  (2005).  F-lll  systems  engineering  case  study.  Wright  Patterson  AFB, 

OH:  Air  Force  Institute  of  Technology.  Retrieved  from  https://acc.dau.mil/ 
CommunityBrowser.aspx?id=37599&lang=en-US 

Shank,  G.  (2006).  Qualitative  research:  A  personal  skills  approach.  Upper  Saddle 
River,  NJ:  Pearson  Merrill  Prentice  Hall. 

Summers,  R.  (1965).  Cost  estimates  as  predictors  of  actual  weapons  costs:  A  study 
of  major  hardware  articles  (RM-3061-PR).  Retrieved  from  RAND  Corporation 
website:  http://www.rand.org/pubs/research_memoranda/RM3061.html 

TFX  contract  investigation,  parts  1-10:  Hearings  before  the  Permanent  Subcommittee 
on  Investigations  of  the  Committee  on  Government  Operations ,  Senate,  88th 
Cong.  1  (1963).  Washington,  DC:  U.S.  Government  Printing  Office. 

TFX  contract  investigation,  parts  1-3:  Hearings  before  the  Permanent  Subcommittee 
on  Investigations  of  the  Committee  on  Government  Operations,  Senate,  91st 
Cong.  2  (1970).  Washington,  DC:  U.S.  Government  Printing  Office. 

Tirpak,  J.  A.  (2015,  June  1)  F-35  block  4  taking  shape.  Air  Force  Magazine:  Online 
Journal  of  the  Air  Force  Association.  Retrieved  from  http://www.airforcemag. 
com/D  RArchive/Pages/2015/June%202015/June%2001%202015/F-35-Block-4- 
Taking-Shape.aspx 

Uda,  R.  T.  (2012).  Exploring  factors  contributing  to  improvised  explosive  device  ( IED ) 
terror  campaigns  in  the  United  States  homeland  (Publication  No.  3530099). 
Available  from  Dissertations  &  Theses  @  Northcentral  University;  ProQuest 
Dissertations  &  Theses  Global  (1085492122).  Retrieved  from  http://search. 
proquest.com.  proxy1.ncu.edu/docview/1085492122?accountid=28180 

Wacker,  J.  (1998).  A  definition  of  theory:  Research  guidelines  for  different  theory¬ 
building  research  methods  in  operations  management.  Journal  of  Operations 
Management,  76(4),  361-3  85.  doi:10.1016/S0272-6963(98)00019-9 

Wicht,  A.  C.,  &  Crawley,  E.  F.  (2012).  Relieving  joint  pain:  Planning  government 

acquisition  of  complex  common  systems.  Defense  Acquisition  Research  Journal, 
79(3),  221-248.  Retrieved  from  http://www.dau.mil/pubscats/PubsCats/AR%20 
Journal/arj64/Wicht_Crawley_ARJ63.pdf 

Whittle,  R.  (2010).  The  dream  machine:  The  untold  history  of  the  notorious  V-22 
Osprey.  New  York,  NY:  Simon  &  Schuster. 

Yin  R.  (1994).  Case  study  research:  Designs  and  methods  (2nd  ed.).  Newbury  Park, 

CA:  Sage. 

Young,  H.,  Vorwig,  P.,  Lieberman,  M.,  Wigginton,  W.,  House,  M.,  Black,  W.,  & 

Maravilla,  C.  (2010).  Aerospace  and  defense  industries.  The  International 
Lawyer,  44(1),  337-353.  Retrieved  from  http://search.proquest.com/docview/ 
645692834?accountid=28180 

Zikmund,  W.,  Babin,  B.,  Carr,  J.,  &  Griffin,  M.  (2010).  Business  research  methods.  (8th 
ed.).  Mason,  OH:  Thomas/South-Western. 


Defense  ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


159 


Scandal  and  Tragedy? 


http://www.dau.mil 


Biography 


Col  Roger  Witek,  USAF  (Ret.),  is  currently  the 
Director  of  Training  and  Services  for  Tapestry 
Solutions.  Dr.  Witek  was  chief  of  Air  Education 
and  Training  Command's  Joint  Strike  Fighter 
Coordination  and  Training  Office,  responsible 
for  the  pilot  and  maintenance  training  programs 
for  the  USAF  and  seven  Partner  nations.  He  has  a 
doctorate  in  business  administration,  is  a  former 
vice-wingand  squadron  commander,  and  is  a  grad¬ 
uate  of  the  USAF  Weapons  Instructor  Course  and 
the  School  of  Advanced  Air  and  Space  Studies.  As 
a  Secretary  of  Defense  Corporate  Fellow,  he  pro¬ 
duced  a  research  report,  “Hard  Skills,  Soft  Skills, 
Savviness,  and  Discipline— Recommendations 
for  Successful  Acquisition"  (available  at  http:// 
dcmo.defense.gov/Portals/47/Documents/ 

Products_Services/SecDef_Fellows/08_09 _ 

Boeing_Report.doc). 

(E-mail  address:  rcwitekl989@gmail.com) 


160 


Defense ARJ,  January  2017,  Vol.  24  No.  1 : 102-160 


